Hacker News Reader

Comment by hughes | Hacker News Reader

It definitely constructs a face, but how accurate is it?

Would be interesting to see a measurement of error against a known result. Maybe try rendering an image of a 3d model of a face, then attempt to reconstruct it from the rendered image and measure the displacement from the original model.

View on HN

It's fitting faces to some kind of assumed model.

I tried it on a few Japanese faces; it basically gave them a Caucasian-looking side profile: big foreheads with over-hanging eyebrow ridges; big noses.

Completely unrecognizable after turning more than a few degrees from front view.

Which is not to flame the project---nice work!---but as a potential improvement for certain applications, it would be useful to simply have these details as parameters that could be interactively tweaked.

Being able to adjust the overall roundness of the face, and prominence of the chin/forehead/nose would probably go a long way.

You could use an expert system to classify the face by race. Then fit it over the appropriate model.

That is the wrong tool for the wrong problem. You don't need to classify by race, and an expert system would be a horrible way to do it.

Just get more Asian people in the training data.

Well, they include an error metric in the paper - basically average vertex distance of predicted face and actual face model (in interocular distance units). They seem to show that they are much better than state of the art with respect to that error, but of course if you want to interpret what that actually means then it's probably easier to interpret the error with your suggestion of actually visualizing it.

If you look at the .obj model file that you can download you can see exactly how accurate it it. The mesh is dense and the detail is not. The detail that it does have however is very plausible.

One of the authors here. You are very right in saying that the there aren't many details. This limitation, we believe, is due to a lack of large, high quality training sets. The data we trained from was very smooth, which means our method is unable to pick out features such as wrinkles and dimples.

What about the "Maybe try rendering an image of a 3d model of a face, then attempt to reconstruct it from the rendered image and measure the displacement from the original model." approach for generating high resolution training data?

That's actually a great idea. Because then you can get a diff between the outputted 3d and the actual 3d used to generate the images, as opposed to what I assume is just diffing the generated profile with a profile shot of the subject.

Reply on news.ycombinator.com