Retargetting Example Sounds
to Interactive Physics-Driven Animations
ACCEPTED TO AES 35th INTERNATIONAL
CONFERENCE
AUDIO FOR GAMES
Overview
We present the results of our method to generate audio in the context of interactive animations driven by a
physics engine, such as those used extensively in games. Our approach consists in retargetting
audio grains extracted from pre-recorded contact sounds to interactive physics-driven
animations.
We first present results of
our approach for resynthesis. In this case, the purpose is simply to reduce the size
of the recording database manipulated during real-time interaction in
video games. The method is applied on recorded
contact events involving different material types (e.g. stone glass,
metal, wood, etc..) and interaction types (e.g. break, hit, roll,
slide). We also demonstrate that our technique is well adapted for the specific case of
continuous-like events such as rolling or sliding. A specific operation is performed
on continuous-like recordings that separates the sinusoidal and the transient parts to allow coding them
separately.
Using this approach, the audio grains can be efficiently
concatenated using the coding structure of user-defined, shader-like
procedures. Similar audio events can be built using different approaches.
Time-scale modification of the original recordings is easily implemented and different examples are given (1).
Besides time scaling, continuous-like recordings are varied by enhancing the sinusoidal part or the transient part extracted in a preprocess analysis step, acting on the granularity sensation of the contact interaction (2).
Finally, variable audio content is appropriately matched to a caracteristic rhythmic
pattern of an available audio recording using cross-correlations calculations(3).
Finally, we present our results
for an
interactive video sequence, similar to a game setting. Here atom
distribution is tailored by parameters from game data. Sounding
material is retargeted to simulated sounding events through the
amplitude of the normal force and relative velocity parameters.
Each sound clip contains the original exemplar followed by the
synthesis result.
Resynthesis Results
Each sound clip contains the ORIGINAL exemplar followed by the
RESYNTHESIS result.
- 'STONE Interactions' : Total
Resynthesis Encoding Size (vs original) = 80%
'IMPACT-like' contacts
'Stone/Break' (30 records): 1034 impulsive atoms
'Stone/Hit' (44 records): 1086 impulsive atoms
|
Stone/Break
1
2
3
|
Stone/Hit 1
2
3
|
'CONTINUOUS-like' contacts
'Stone/Roll' (3 records):
276 impulsive atoms
-
313 continuous atoms
'Stone/Slide' (17 records):
1250 impulsive atoms
-
1212 continuous atoms
|
Stone/Roll 1
2
|
Stone/Slide 1
2
|
Stone/Gun 1
2
|
- 'WOOD Interactions'
'IMPACT-like' contacts
|
Wood/Break
1
2
|
Wood/Hit 1
2
|
'CONTINUOUS-like' contacts
- 'GLASS Interactions'
'IMPACT-like' contacts
|
Glass/Break
1
2
|
Glass/Hit 1
2
|
'CONTINUOUS-like' contacts
Results of Similar Patterns Synthesis
Each sound clip contains the ORIGINAL exemplar followed by the
SYNTHESIS result.
- TIME-SCALE Modifications : according TM factor
|
TM=0.3
(shrinking): 1
2
3
|
TM=1.3
(stretching): 1
2
3
|
- CONTINUOUS-like contacts variations :
independent time-scale (TM) and granularity (Gran) modifications
|
1
: TM=1 - Gran=0.45
2
: TM=1 - Gran=4
3
: TM=0.3 -
Gran=1
|
1
: TM=0.4 - Gran=2
2
: TM=1.2 - Gran=0.08
3
: TM=1.4 - Gran=0.4
|
- Audio Content Variations : only the rhythmic pattern of the original recording is kept
Results of Retargetting to an
Interactive Physics-Driven Animation (game-like setting)
