Hacker News Reader

Comment by savanaly | Hacker News Reader

I'm curious, since the facial recognition algorithms they are testing are created with ML, would adding the instances of someone wearing these glasses to the training set result in a version that wouldn't be fooled? Or is the method something the ML algorithms are inherently weak against? Is this an exploit that could be patched, in other words?

View on HN

Adding some instances of these glasses to the training set would probably fix the model for these glasses, but then it might break for lipstick, hats, scarves, other glasses, or what-not.

The most interesting thing to me about this is that the models could be so badly fooled while obscuring such a small amount of the face. It shows the limits of solving problems by training zillion-parameter models to categorize zillion-dimensional data points: they won't necessarily be doing what you think they are, and may go completely off the rails when presented with data not close enough to their training sets.

> ... so badly fooled while obscuring such a small amount of the face ... may go completely off the rails when presented with data not close enough to their training sets.

Older research has looked at this, in the context of neural networks in self-driving cars. The nets take the easy way out: use just the most significant features (like road markers) to solve the problem. However, training sets may not contain roads where snow or fog is obscuring these significant features, which makes the nets fail, because they haven't learned to rely on other features too. In contrast, human vision is remarkably robust.

http://repository.cmu.edu/cgi/viewcontent.cgi?article=1176&c... [pdf] "FeatureBoost: A Meta Learning Algorithm that Improves Model Robustness"

Thank you! My knowledge of how these things work is somewhat general and out-of-date. I think I remember reading about FeatureBoost back in the day, and will try to refresh my memory of how it compares to max-margin methods.

You can adapt your training of an ML model to defend against these attacks. The paper even mentions that they will investigate this in the conclusion.

Reply on news.ycombinator.com