Facial hair tracking for high fidelity performance capture | Hacker News Reader

Facial hair tracking for high fidelity performance capture

studios.disneyresearch.com

130 pointsstaindk35 comments

I've linked to the publication but be sure to check out the video they've embedded.

Out of the handful of new publications they released today this one was the most interesting to me - I guess it's because this problem is in many ways more focused and less complex (seeming?) than the others (I'm not an ML/neural networks buff but do find it all interesting).

Very cool stuff.

This is a bit of a tangent, but why are the captures all toothless? Is there something in teeth that makes it hard to capture them? Or is it a conscious choice to remove them? I have the same question regarding the eyes.

My guess is that the teeth isn't visible throughout all the frames, so it's hard to reconstruct accurately in 3D. In fact the parts that look like gum and the tongue don't seem to be correct, far from the reconstruction quality of the face itself.

They're not? You can clearly see the teeth in the frames where they are visible in the raw. Teeth are just not commonly seen outside unnatural facial movements.

At 1 minute in the video in the mesh reconstruction, the gums / tongue are modeled, and there's clearly no teeth

The short answer is that they're only showing meshes that are deforming. Teeth are rigid and would be put back in based off the jaw and skull position. If memory serves, eyeball position tracking isn't great with this approach, it's not all that hard to add back in, and they often want to cheat eyelines anyway.

I think pfranz is on the right track - in one of the other publications they released today they show off how their tech does jaw positioning based on certain constraints (it seems to know where the jaw hinges and how the jaw bone interfaces with the skull).

I love whenever Disney makes things like this public. Last one I saw was about snow rendering, some months before Frozen was released

Although this link isn’t it, I’m anticipating a time when facial hair tracking may become a proxy measurement mental health well-being - ie individuals who are usually clean shaven all of a sudden start having a five o’clock shadow.

What about people who usually have beards who suddenly become clean shaven?

When I do that I'm usually up to no good.

Signs they've got interviews scheduled and will be leaving soon.

“Based on our StubbleScan report, your supervisor has been notified!”

Heh. Always means I have an upper respiratory infection.

I read your comment, and rubbed my chin, noticing the unusual scratchy beard starting to grow.

I think I need a day off.

Are you suggesting direct integration of the StubbleScan into PeopleSoft for automatic scheduling of mental health days and/or automated referral to the Employee Assistance Program?

I suspect this is likely to be met with significant kickback, but I'm glad there are people publishing and researching ideas outside the cutting edge, yet easier to get grants for, ML/DL stuff. Not to mention that many researchers have noted the dubious quality of many papers in big ML conferences and no I don't believe they complain because theirs didn't pass peer review.

Reading about CS research here, as well as in other media, has become akin to TV zapping and always ending up with the same type of show on every channel.

> I suspect this is likely to be met with significant kickback…

Why do you think people would have an issue with them tracking us furry humans?

I think the implication is that ‘women don’t have facial hair’ and that’s going to be the reason for kickback.

I've seen plenty of women with facial hair.

Basically all women have facial hair, unless they've done something to remove it. I'm not sure if this technique would work on it though, it tends to be difficult to see clearly from a distance.

I was talking about being a party pooper for not getting very excited by ML research.

This is graphics research done by a private company, not funded by grants. Siggraph is one of the pickier CS conferences, even when it comes to applied ML. I'm not sure I understand why you think this would get any kickback, or what you mean by that?

I supposed my original message might come off as anti-intellectualism towards ML research. This is personal but I think AI research* has become incredibly over-hyped, so for me reading about contributions to other fields is something more exciting now.

*referring mostly to applied disciplines. I cannot speak for theoretical ML research as I have not looked into it in depth.

This article assumes that only male actors have facial hair. That's rather nonprogressive of Disney.

Lol.

Real talk, most people have some amount of facial hair, male or female. A lot of work goes into recreating it as it adds a lot of realism to a render

Disney animation and Pixar are both well known for doing that work, adding subtle hair to female faces in the movies they produce.

No, it doesn't. Eyebrows are considered facial hair for this. It mentions that currently males need to shave off their mustache or beard, but that is different from saying women don't have facial hair. Most people haven't shaven off their eyebrows.

This text from the fine article:

> actors are asked to shave clean before face capture, which is very often undesirable.

Had the additional word "male" yesterday:

> male actors are asked to shave clean before face capture, which is very often undesirable.

They actually went and changed the text! That is quite typically Disney, for better or for worse. The original text is luckily preserved at the Internet Archive:

https://web.archive.org/web/20220724134011/https://studios.d...

I’m confused by your comments, which wording do you think it should have? Like parent comment already said, the presence of the word ‘male’ in that sentence does not imply an assumption that all facial hair is male; it can legitimately mean that only the male actors typically need to shave, which is usually true. Facial hair growth has sex based differences in humans, so I don’t see a problem with having the word there, nor with removing it. Why do you think it’s a problem either before or after, and why do you suggest that changing the text to fix the very issue you called out is a bad thing?

It should probably have whichever text is most accurate to whatever actually happened or is happening.

So if they did/do actually ask only the male actors, it should probably say that. If they're asked only based on the characteristics of their facial hair, it could say that.

Sure, sounds fine. That doesn’t explain why either choice deserves criticism.

How unprogressive of you to focus on male/female, while neglecting people with facial deformities or people who have excessive hair growth in non-traditional facial areas. You oppressor.

I'm glad you're injecting these considerations into every possible topic. We need more of it. That way I have someone to accuse of being inconsiderate, by finding the next category that we'll divide people up by.

How unprogressive of thou to assume that I prefer being addressed as "you"! I shall not be oppressed into conforming to tyrannical social definitions.

Reply on news.ycombinator.com