Work in progress Automatic Lip Sync

This is a quick test of the first hackish pass at doing automatic lip sync based on phoneme recognition from PocketSphinx.

I had reluctantly set the idea aside due to poor results reported by UE4 forum user ShaneC, who wrote a nice wrapper for PocketSphinx as a plugin. After some fiddling around and much searching of usergroups about Sphinx, I was able to determine that passing in a blank option for the phoneme recognition file makes it work pretty reliably for some reason.
Playback of the visemes is a bit choppy right now for a number of reasons, mostly to do with just because it hasn’t had a pass to make it look nice, and because I’m not passing through the information about how long each phoneme lasts.

Needless to say this is completely pre-alpha and doesn’t represent the final result, just an extremely early first test.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s