Hacker News Reader

Comment by cs702 | Hacker News Reader

The key innovation is an accurate, reliable method for rotating faces so they're 'looking straight at the camera' before feeding them to a deep neural network. They call this 3D photo rotation process "frontalization." Figure 1 on page 2 of the paper shows at a very high level how this is being done. Very nice!

View on HN

Just a little background the paper itself doesn't provide:

The 3-d modeling and rotation is building on the work Yaniv did as part of Face.com (face recognition startup), which was acquired by Facebook.com. Studied here: http://vis-www.cs.umass.edu/lfw/results.html

Also Marc'Aurelio was just hired away from Google and is a deep learning expert.

Actually, that's only one of the contributions, and I'm not so sure it's the "key innovation". Every other recent face recognition method also tries to do some kind of alignment to make faces more similar in pose/expression/lighting prior to classifying them; and of these, several also fit faces to a 3-d model to rotate to frontal (with varying quality).

See my other comment for my guess on what's actually providing the boost: https://news.ycombinator.com/item?id=7393378

Yes, other attempts "also fit faces to a 3-d model to rotate to frontal (with varying quality)," as you put it, but this method for rotating faces appears to be superior -- more accurate and reliable.

It looks like the main contribution to me :-)

Yeah, generally speaking for object detection - Scale, position, rotation: typically you can identify something based on 2 of these really efficiently (for example, wavelets give you a transform that is sufficient for object identification under scale and position transformations in O(N) operations).

If you'd rather have orientation identification (ie. rotation angle) and scale in that mix but don't care about position, the radon transform is nice and easy to work with.

But beyond inverting 1-2 key transformations, one usually has to pay a pretty hefty computational cost which often precludes online (near real-time) use.

Reply on news.ycombinator.com