Speaking with Hands: Creating Animated Conversational Characters from
Recordings of Human Performance
"That
was ugly, dude! On this run you forgot to set up your jump."
Abstract:
We describe a method for using a database of recorded speech
and captured motion to create an animated conversational character. People's
utterances are composed of short, clearly-delimited phrases; in each phrase,
gesture and speech go together meaningfully and synchronize at a common point of
maximum emphasis. We develop tools for collecting and managing performance data
that exploit this structure. The tools help create scripts for performers, help
annotate and segment performance data, and structure specific messages for
characters to use within application contexts. Our animations then reproduce
this structure. They recombine motion samples with new speech samples to
recreate coherent phrases, and blend segments of speech and motion together
phrase-by-phrase into extended utterances. By framing problems for utterance
generation and synthesis so that they can draw closely on a talented
performance, our techniques support the rapid construction of animated
characters with rich and appropriate expression.
Publications:
Matthew Stone, Doug DeCarlo, Insuk Oh, Christian
Rodriguez, Adrian Stere, Alyssa Lees, Chris Bregler. Speaking with Hands:
Creating Animated Conversational Characters from Recordings of Human
Performance In SIGGRAPH 2004.
