Automatic synchronization of background music and motion in computer animation

Abstract

We synchronize background music with an animation by changing the timing of both, an approach which minimizesthe damage to either. Starting from a MIDI file and motion data, feature points are extracted from both sources,paired, and then synchronized using dynamic programming to time-scale the music and to timewarp the motion.We also introduce the music graph, a directed graph which encapsulates connections between many short musicsequences. By traversing a music graph we can generate large amounts of new background music, in which weexpect to find a sequence which matches the motion better than the original music.