Workflow
A suitable wav file was downloaded from The daily wav. Lipsync was done in Papagayo, and exported to a Moho .dat file. The mesh was modelled in Makehuman; basically, I just cranked up the ethnical morphs for central african and nordic types (Kenyan father and blond mother). The mhx file was imported into Blender. Then the lipsync script was started, and the Moho file was imported.
Papagayo generally produces very choppy animations. The ipo curves can be smoothed by pressing the Damping key; it ensures that the shapekeys do not change faster than the maximal slope per frame. Then I added blinks and some random movements of the head, shoulder and brows to create some impression of life. Finally, sound was added in Blender's sequencer.
Some of the things that I learned:
1. There is a difference between wav and wav. The wav file from Daily wav could not be loaded into Papagayo or Blender. Therefore I loaded it into Audacity and exported it as a new wav file, which could be loaded.
2. It is easy to forget to load the pydrivers.py script.
3. The shape keys sometimes lose contact with the rig, and freeze into some ugly pose. I don't know if this is some Blender bug, or if the MHX import script does something illegal. Anyway, I managed to reconnect shapekeys to the rig with the following steps:
a. Unlink the action from the armature.
b. Reset (Alt-G) all bone positions.
c. Open an IPO window and select one of the shapekeys.
d. Connect a shape (one is enough) to its driver (Alt-N, return).
e. Link the action to the armature again.
4. The default frame rate is 24 fps in Papagayo but 25 fps in Blender.
Hello Thomas..I've been working on the same problem since fall of 2007. The ugly shapes, I believe, are the sum of all the shape keys for that vertex group. The shape key (norph target) needs to be set to 0 when initialized; I just do it with the slider. The more difficult problem is the need to have a 'mouth closed' shape separate from the basis shape. This is because animations need to close their mouths between all words and some consonants. We real-world humans don't do this. Google 'Preston Blair visemes' for a full explanation.
ReplyDeleteThis is my first lipsync attempt, and it shows. I might go back and redo it, but not in the near future. OTOH, there might a pedagogical point to set the target so low that anybody can surpass it :-)
ReplyDeleteEnglish is the worst language for lipsync, words are written like that and said different, example fire->fa-i-er.
ReplyDeleteThings may be easier with some other languages like japanese, languages with latin origins(italian,spanish) or german.
If you continue this script, it will be interesting to have custom mapping of words to phonemes. It will be easier to map various languages to the same animation.