speech synthesis

phoneme opcode speech synth video avatar (Composition by dust)

Author: dust
License: Public Domain
Date: 2011.05.12
Compatibility: 10.6
Categories:
Required plugins:
opcoder

so not really sure what to call this. its basically the speech synth plugin with phoneme opcode output and rate input. the phoneme opcode is a code that represent a phoneme.

this patch maps the phoneme opcode out to video clips of the phonemes. years ago i recorded this actor reading the opcode and chopped it up... while i was searching for cross modal connections between vision and sound. http://en.wikipedia.org/wiki/McGurk_effect

turned out to be more of a max headroom type of effect i guess. simple one to one map. type in text watch video speak the text.

this is done all the time with animations though normally just a few key blend shapes with various weighting can produce most lip syncs. so normally people don't actually produce 42 different shapes for each phoneme.

either way you have this output that can be mapped to what ever you like with a rate input to speed up or slow down the voice.

twitter rss->speech synth->face animation

gtoledo3's picture

Twithead - Run 1. Twitter Realtime Animation from George Toledo on Vimeo.

This is something I put together that uses the Kineme3D, with facial expression triggered by sound created by feeding a string generated by an rss feed to a speech synthesis patch, as well as "tweet" number.