Aqsis skin shader improvements

Yesterday I've committed some important improvements of skin shader for Aqsis.
Because the accuracy of new mesh topology, we can use an internal baking engine, based on vert shader. It require just few seconds to be generated, and the results are enough good to be used as scattering light map(see image below).

Another good notice is the new rendering pipeline finally remove (or better, make almost invisible) the "famous" seams artefacts. Further improvements are possible (precalculated AO, better scattering by channels, separated eye geometry and materials) but the level is already satisfying. The image below is just a quick test with "Caravaggio's lights", rendered with Aqsis 1.6:

Z up

The last week I have struggled with rotating the mesh 90 degrees. In Blender, the z-axis points up, which is very natural for someone with a science background. However, in MakeHuman and many other programs like Maya it is the y-axis that points up, which means that the character comes into Blender lying on his back. To correct for this, I have worked on an option that loads the character standing on his feet instead.

This sound really easy; instead of putting a vertex at (x,y,z), put it at (x,-z,y) instead. But not so fast. Shape keys and joints also need to be rotated in the same way. After this fix, we are ready to load the mesh with his head up.

But what is this? The circles around the mesh have the wrong rotation. The problem is that several meshes are imported. Apart from the main mesh called "Human" there are several display objects hidden on layer 20, and they should not be rotated. To fix this, we decide to only rotate the mesh if its name is "Human", and load the mhx file again.

More problems! I changed the shape of the panel driving the facial keys to something more intuitive; I will discuss facial animation and lipsync some other day. The most striking thing is that the little face is upsidedown. This is not how it is supposed to be, and not how it comes out if we load the mesh without rotation.

The problem has to do with roll angles. In Blender, each bone has its own coordinate system with the y-axis pointing from head to tail. A priori, the only thing we know about the x- and z-axes is that they are perpendicular to the bone. The direction of the z-axis is defined by the roll angle. I don't really know the exact definition of the roll angle, but this is not important. What is important that the roll angle is chosen in a predictable and consistent manner. It the roll angle is not given (which is the same thing as setting it equal to zero), the rotation of some bones relative to the armature differs between Y up and Z up building.

The solution is to provide a table of the roll angles for all bones. For bones pointing left-right or up-down, define the local z-axis to point forward, to a point far in front of the character. This is not a good definition for bones in the front-back direction, because the y-axis already points forward, so these have z pointing down instead. In this way, all bones have the same orientation relative to the armature, for both Y up and Z up.

At this point I was ready to commit, when I realized that things do not work when IK is turned on. Driven shape keys are implemented by Py-drivers, which give great flexibility. To work, the file pydrivers.py must be loaded into a Blender text window before the mesh is imported (this is not my idea, but just the way Py-drivers work). The rotation of the upper left arm around the x-axis, say, is given by the expression

Blender.Object.Get('HumanRig').getPose().bones['UpArm_L'].quat.x.

Alas, this value is only updated when the bone is rotated actively, and not when it moves in an IK chain. So I changed the implementation of shape keys driven by bone rotations to be independent of python, and now things work as expected. Facial shape keys are still implemented by Py-drivers, but that is ok because the bones in the face panel are not part of an IK chain.

Finally time to commit. Suddenly the makehuman site refuses to let me do so, despite having successfully committing to svn several times in the past. Don't know if I have forgotten how to log in, or if there is any problems on the MH side. I will make another try at my work computer tomorrow, but for the time being this code only exists on my computer.

Correction of targets

I'm trying to correct artifacts with a python script. I have just completed the right eyelash (as in the pictures) and I'll extend the work for the left eyelash, browneye, eyes, etc

Working on the new base mesh: new base texture.

I've just completed the new color texture, using latest UV layout. Reflection and sss textures are still to do, but they are based on this one, so it should be a quick task.
As soon the textures are finished, I need to modify some things in Aqsis plugin, in order to fix some recent problems related to the changes of mesh.

Working on the new base mesh: UV layout

New UV layout. Less cuts, less distortions, more Gimp-like texture. In future we will add a dynamic UV system, so the layout will be changed by morphing.

Now I'm working on the base texture.

UV test in realtime view:

 

Working on the new base mesh: feet

I've just completed the new base mesh. Thomas will test the conversion of old targets as soon as possible. Then I'll start to work on the UV layout.

Problems in the targets conversion

Today I've received this email from Thomas Larsson (thx!). It's about the targets conversion from old mesh to the new one we are working on (see, for example: working-on-new-base-mesh-eye , working-on-new-base-mesh-body ,  eyeball-test ):

Hi Manuel
I have tried to run my program on the new mesh, and have encountered some problems.

1. The old eye consists of an eyeball and an iris, the new eye of two concentric eyeballs. The program works if I remove the iris material and assigned everything in the eye to the same material. May cause problems later.

2. The eyelashes have changed shape, which my program does not like. The new ones look much better. I remodelled the old ones, to stop my program from complaining. May cause problems later.

3. The eyebrows have changed shape. Since the brows move with the head, I simply assigned the skin material to the new brows. This should be OK.

4. The toes have changed shape. In particular, the toe joints have moved so far that the program finds no suitable faces. One should modify either the new or the old mesh, so the toes and toe joints are at the same place. 

I edited the toes in the old mesh. After that the program didn't complain any more, and I could test it on the African child. Toes look good, but there are some issues in the head:

Right now I'm working on the new mesh, remodelling feet and toes joints, and trying to improve  the compatibility with old eyelids. As soon as the new mesh will be finished, with an acceptable targets conversion and a new UV layout, we will release the alpha 4 of makeHuman. I guess the conversion will never be perfect, so some artefacts are expected in next release (they will be fixed, in the worst case, by hand, later). Stay tuned.

Collision Detection

Collision detection for hair strands. That's a tough nut to crack! Makehuman does not have it yet, but in the svn you can find workable python codes that can be used in Blender (namely makehair). I used an old octree code donated to Manuel and I modified it a bit. But Octree alone will not save you of all the troubles. Most of the difficulties arise in things like: 

1. Starting after what length of the strands should one start doing a collision detection
   
2. Octree is used because it makes the method inexpensive, but given bad resolution at certain mesh levels it can give us really poor results.. so we need extra algorithms for that (like in case of Bezier hairs, after a collision is detected and hair is deflected one should skip the next control point and not look for collision there..etc.)
3.  Cup-shaped meshes poses the biggest problem when hair falls into them (like the collar bones).. even with the best algorithms I have written so far, you can get some strands passing through the body.
   

But in general, I am satisfied with the results. Hair looks more natural and follow the tangent of the upper body and the neck. Now if I could figure out... how to exactly solve the problem with the hair ends (see the figure) after I make the collision detection!

Bones that bend

Today I want to discuss a technique for improving the deformation of a mesh: shape keys driven by bones. Consider the MakeHuman mesh, imported into Blender using the MHX format that I described a few days ago. When the arm is bent, the mesh collapes into itself at the inside of the elbow, in a quite ugly way. Even if you can evade this problem to some degree if you paint weights more carefully than I have done, this is a quite generic problem, not specific to MakeHuman or Blender.

Now driven shape keys come to the rescue. The shape key BendElbowForward_L is not pretty; it looks like our character has a serious health problem. However, this shape key is not seen when the arm is straight. Instead, it is connected to the rotation of the lower arm bone LoArm_L along its local X azis. The influence of the shape key varies linearly from 0.0 at 0 degrees to 1.0 at 90 degrees. Now when we bend the arm, the shape key cancels the collapse of the mesh, making the elbow deform much nicer. Perhaps the shape of the elbow is still not anatomically correct, but it looks a great deal better than before.

As a bonus, I added a little biceps bulge to the BendElbowForward_L key, although it can barely be noticed. This illustrates a fundamental limitation of my approach. I have painted weights and shape keys on the base mesh, which means that the biceps of every character will bulge the same amount. However, the biceps should really bulge more on mr Macho than on Anna Anorectic. Eventually, deformation should be handled by the pose engine with more detailed knowledge of human anatomy; the skinning done by the exporter is only a quick-and-dirty solution. But we probably need to "bake" deformations to shape keys driven by bones before export, because that is what Blender (and other 3D packages) understand.

MSVC 6.0 build for people who have problem with makehuman installs

I just learned (thanks to Manuel and Marc) to compile Makehuman properly. I was too lazy to write an installer, so I just zipped the whole build. I am using Visual C++ 6.0 for compiling and I used python 2.5 for it (don't worry about these details, all necessary libraries are in the .zip file).

I learned that some users have had problems installing makehuman, especially those who needed to install a Visual Studio Redistributable Package. I know Microsoft had made our life more difficult after MSVC 6.0 so I compiled it with MSVC 6.0. Most user's would not require any redistributable package and builds from MSVC 6.0 is known to work in all windows platform Win95 and above. 

Here you can download a .zip MakeHuman using my build:

In case ifile.it does not work, I uploaded copies in:

drop.io

Mediafire

Please comment if your installs were successfull or not or if there is any other technical matters. We are happy to have anyone to test this in their machine.

Blender export with the MHX format

MakeHuman is a great program, but it does not exist in isolation. After designing your character in MakeHuman, you probably want to export it to some other 3D package. I have been working on an exporter specifically designed for Blender, because that is what I know and have access to (ok, I did play around a little with 3ds max 4.2 a long time ago, but that is history by now).

MakeHuman has of course already exporters to the standard OBJ and BVH formats, but they leave a lot to be desired. For one thing, the OBJ mesh and the BVH armature do not end up at the same place, at least not in Blender. Moreover, a barebone rig is not so useful for the animator – she typically wants more control (like arm and leg IK), a skinned mesh, and shape keys. The MHX (MakeHuman eXchange) format attempts to address these issues. I will discuss some specific problems with rigging and MHX export in a series of posts, and how things are done (or in some cases, why I have failed to implement them). However, in this initial post I will describe how to use MHX to get a rigged character from MakeHuman into Blender.

Important caveats: I have tried this with the old mesh. Since the new mesh is just around the corner (touch wood), this version has not yet been committed to the trunk. Also, I use Blender 2.49b. Trying to import mhx files into Blender 2.49a and before will probably be problematic, due to some bug in the treatment of driven bones. And the code will not work in 2.50, which is quite different from a Python point of view.

On the MakeHuman side things appear to be the same as always: create the character and press the export button. However, the makehuman/exports directory contains a new file with extension mhx, in addition to the usual obj and bvh files. It is this file that we will import into Blender.

First we need to do some preparation. In the makehuman/utils/mhx directory there is a file called ”mhx_import.py”, which you need to copy into your Blender scripts directory. When Blender starts up, there is a new entry in the File > Imports menu which says Makehuman (.mhx).

It may happen that you don’t have privileges to copy files to the Blender scripts directory; this happens on my Ubuntu box at home. If so, just open the mhx_import.py script in a text window in Blender and execute it from there. Either way, you should now be prompted with a blue screen. Here you can select a number of options:

1. Should arms, legs and fingers be controlled by forward (FK) or inverse (IK) kinematics?

2. Should shapekeys be loaded? This takes some time and the blend file becomes big, but it is worth it.

3. Should the imported character replace the whole scene? It is best to hit Ctrl-X and import into a pristine blend file.

4. Do we want display objects for bones?

5. Select the directory where the import script should look for textures.

After these selections are made, we press the ”Load MHX file” button, and select the mhx file in the MakeHuman exports directory with a file browser.

After a number of seconds, you should see something like the image to the left; the character is loaded into Blender, with a panel on top of him. We are now ready to start posing. For some reason, the control bones do not show up on my system until some part of the armature, e.g. the panel over the character’s head, has been selected. The panel which controls the facial shapes more or less follows what Jason Osipa, in his famous book ”Stop staring”, calls the simple setup.

I like to move the morph panel into a separate window, which is set in wireframe mode (z-key).

And here is the result of five minutes worth of posing.

Keypoints detection: modelisation of features

I am trying to find a good and simple modelisation for the features extracted from 3D scans. After several tries the best modelisation seems to be given using the skew normal distribution.

Here are some examples of feature distributions and the corresponding models we get for different keypoints:


As it can be seen, it captures more or less correctly the global shape of the distributions.

Thank you to my lab mates for the interesting conversation about this modelisation issue.

MakeHuman in crime scene 3d reconstruction

One of the possible uses of MakeHuman is 3d modeling of victims or aggressors in the crime scenes 3d reconstruction;

I had opportunity to talk about this argument recently, at the first "forensic balistic course" organized by ForenLab (Ferrara - Italy).

For this job, most important is the ability to make a precise metric modeling of the human body, for wich will be very useful the new measurement tools that Manuel is preparing.

On the body mesh can be reconstructed set of reference points at the points of passage of bullets, thus rigging and moving the human model can try to reconstruct the trajectories...(I apologize for my bad english)...

Working on the new base mesh: body measurement loops.

The new makehuman mesh (to be reelased in the incoming alpha4) will support edge loops for standard body measurements.

Eyeball test

The eyeball is made of four separated parts:

iris, cornea, sclera and a little black square behind the pupil.

There are three materials: iris has an UV mapped photo texture, cornea has a marble texture to simulate the veins, sclera has the same texture of cornea, applied to the normals only (to give a bit of bumps on the transparent sphere).

Every effect is made tweaking the simplest blender settings, and to be honest most of the realism comes straight from the iris photo (it has most of the reflections). This may be a limit, but it permits to give away a good compromise between quality and rendering times.

When in MH, the iris photo may be easily replaced to allow further customisation (the one I used is from my left eye, black spot included).

In this render only one light is used (of type lamp, almost default settings).

Any critic is welcome, and any hint on how to get transparencies and reflections without raytracing would be fantastic.

Working on the new base mesh: the eye.

Better topology of eye, as suggested by Steven Stahlberg and others. Instead the absolute loop around eye areas, the eyelids continue along an external line that go out from the outside corner, as showed in the picture below.

I've added the eye's canthus and remodelled the infraorbital line as well.

This is important in 3d models of children and old people, and it's very noticeable in Asian eyes.