Hacker News Reader

Comment by goatcode | Hacker News Reader

> And the fact is that humans can't reliably identify emotions from photographs, so there's zero chance for AI.

I believe there are a lot of examples of AI doing things that humans can't. Typically it's a matter of scale, but sometimes it's a matter of some correlations being beyond the basic capabilities of humans. I could be wrong, but I'm not sure that this is the best metric for whether AI could do something.

View on HN

First, you need reliable training data to begin with. This can't be supplied by third-party labeling of faces, so it would need to be done with a dataset of self-reported emotions, but self-reported emotions also have tons of pitfalls well documented in the literature.

But second, the correlations of individual facial muscle contractions with emotions has been extensively studied and it's far noisier, inconsistent, or completely devoid of a signal than many people assume. In academic terms there's no such thing as a reliable emotional "signature" to be gleaned from facial muscle activation.

So the point is, it appears that the raw data simply isn't there for the AI to detect patterns that humans can't. Detecting emotions requires far more data points outside of facial muscle activation -- such as the ones I listed.

> This can't be supplied by third-party labeling of faces

Yes it can. You get enough labelling and it overcomes the unreliability of detection by any single human.

Get 70 people to label the same face. A random distribution over the 7 Ekman emotions[1] will give 10 each, and any non-random variation from that is a signal. Do that over enough faces and you'll get something to train on.

(Also, no reason why it needs to be face pictures. It could be 2 second video snippets for example).

[1] https://www.paulekman.com/blog/darwins-claim-universals-faci...

No, because the fundamental problem isn't the unreliability of detection by any single human.

The fundamental problem is that multiple emotions can result in the same facial expression, and the same emotion can result in multiple facial expressions.

It doesn't matter how many people label a face. It won't get over the fundamental issue that there is no 1-1 mapping between emotions and facial expressions.

> It won't get over the fundamental issue that there is no 1-1 mapping between emotions and facial expressions.

Actual experts disagree. For example Ekman, for the basic emotions at least (see the link I posted above). To quote:

In the late sixties, Izard and Ekman in separate studies each showed photographs from Tomkins’ own collection, to people in various literate cultures, Western and Non-Western. They found strong cross-cultural agreement in the labeling of those expressions. Ekman closed the loophole that observing mass media might account for cross cultural agreement by studying people in a Stone Age culture in New Guinea who had seen few if any outsiders and no media portrayals of emotion. These preliterate people also recognized the same emotions when shown the Darwin-Tomkins set. The capacity for humans in radically different cultures to label facial expressions with terms from a list of emotion terms has replicated nearly 200 hundred times.[1]

The FACS coding system[2] has pretty good experimental support, across multiple cultures. As noted on this article, it can distinguish between fake smiles and real smiles etc. See https://en.wikipedia.org/wiki/Smile#Duchenne_smile for more on this.

Even if you disagree with that and think that motion is required (which is an argument that has some validity) there is still no reason to think a computer system can't do it.

[1] https://www.paulekman.com/blog/darwins-claim-universals-faci...

[2] https://en.wikipedia.org/wiki/Facial_Action_Coding_System

Actual experts today disagree with Paul Ekman -- the book I linked to in my root comment provides an extensive discussion of this.

Ekman was at the forefront of creating facial emotions expression research, but while I do believe he convincingly showed that there are basic instinctual, stereotypical, cross-cultural facial expressions, the field now generally accepts that FACS coding does not map reliably to emotions in real-life contexts.

In other words, just because a smile is a cross-cultural instinctual display of happiness does not mean a given individual who is happy will be smiling, or that a given individual who is smiling will be happy.

A 1-1 mapping isn't required, IMHO everybody involved would consider a probability distribution as a reasonable expected output, and as long as that probability distribution is reasonably close to reality and is substantially different from the prior, output like that is useful signal and would count as the system working.

The same applies for labeling, if a particular facial expression can occur in an indistinguishable manner because of three different causes, that's okay; there's nothing wrong with a gold standard label in training data marked as "we've observed that 50% people had this expression because they were angry and 30% because they were in pain, but no happy or sleepy people had an expression like that".

Furthermore, for the purpose listed in the article, you don't need to determine an absolute value of some emotional state, but you need to detect large shifts in it (i.e. you get an baseline that implicitly adjusts for part of individual and cultural differences) - i.e. whether the audience (in aggregate!) now looks significantly more frustrated than the same people looked 30 minutes ago.

Unsupervised learning is really good today, about 1-2% under supervised learning. You can also cross correlate text, audio and video. If there's signal in the data, then a model can learn it, and you don't need so many labels.

The signal is facial expressions. You're missing the “map facial expressions to emotions” step.

Your argument is basically that since something is hard it is impossible?

No, my argument is that if something is impossible it's impossible. I said "the raw data isn't there", not that "the raw data is hard to interpret".

> self-reported emotions also have tons of pitfalls well documented in the literature

That's interesting. Where can I read about these issues?

This is a common misperception about AI and ML. Most leading voices in ML will tell you that--unless you have a very good reason to believe differently--ML is not the tool to turn to if you need better than human accuracy. That also comports with my experience in the field over the last 6 years.

What do you mean by this? This seems like a vast sweeping generalization. There are lots of areas where ML already outperforms human accuracy with open and available tools, are you saying these should be avoided unless there is a very good reason to use them?

How do you know it is accurate if you cannot tell yourself? And the ML learned it by your expectations. Without accurate verification you fundamentally have GIGO.

In this paper (https://www.nature.com/articles/s41551-018-0195-0) Google predicts patients' sex with 97% accuracy from images of their retinas. This was thought to be impossible.

I think this is a pretty decent example of a Neural Net becoming better at something than humans are.

Reply on news.ycombinator.com