|
View your shopping cart.
Recent topics
Recent Comments |
phoneme opcode speech synth video avatar (Composition by dust)
so not really sure what to call this. its basically the speech synth plugin with phoneme opcode output and rate input. the phoneme opcode is a code that represent a phoneme. this patch maps the phoneme opcode out to video clips of the phonemes. years ago i recorded this actor reading the opcode and chopped it up... while i was searching for cross modal connections between vision and sound. http://en.wikipedia.org/wiki/McGurk_effect turned out to be more of a max headroom type of effect i guess. simple one to one map. type in text watch video speak the text. this is done all the time with animations though normally just a few key blend shapes with various weighting can produce most lip syncs. so normally people don't actually produce 42 different shapes for each phoneme. either way you have this output that can be mapped to what ever you like with a rate input to speed up or slow down the voice. |
here is the video avatar with the needed plugin and video avatar. just run the decoder file after you installed the plugin. (edit new version attached)
• edit
i noticed this example patch's asset paths didn't have a ./ in them. i have fixed all 40 phoneme clips so this example number 2 should run in lion.
so i tweaked this out a bit to give a more smooth natural feeling. i kind of liked the glitched out stutter but doesn't really prove this example. so i'm including this revision.
in this patch i'm using the phoneme opcode as an external time base to the video clips. so now the video is playing at the same rate of speech as the synth and provides an almost transparent talking video avatar. what i mean by transparent is you cant see that this is composed of phoneme clips as the video looks natural. the synth on the other hand has come along way but i feel still needs a bit of work to make this sound more natural.
attached is a list of phoneme symbols and code. it will help if you want to make your own in 3d or something. also see the new clipped video examples you can use them as a base to draw 2d mouths if you want to do cartoon effect.