Hacker News Reader

Comment by thedevil | Hacker News Reader

In my state, there's a (fast) road that turns down under a bridge. From a distance, it kind of looks like I'm going to crash into the bridge.

Even though I know the road wouldn't drive directly into the bridge, I slow down a little and look carefully to make sure I'm actually not going to crash into the bridge.

When my perception doesn't fit my internal model, I gather more data (look at different parts of the bridge and what other cars are doing), or transform the data (ie turn my head slightly and look at the bridge and road from different angles)

Edit: Likewise, when someone's tone doesn't match their words, I gather more data (look at their body language).

Have any researchers experimented with neural nets to do the same? I haven't noticed any posts here about that.

View on HN

There was a paper that claimed those sorts of rotations and movements can help alleviate adversarial examples:

https://arxiv.org/abs/1707.03501

However, OpenAI quickly refuted it by creating adversarial examples that continue to fool the classifier even when rotated, scaled, etc:

https://blog.openai.com/robust-adversarial-inputs/

So it looks like there's no "easy" way out here. Multiple types of sensors may help, but it seems likely that it will still be possible to construct examples that fool network over all sensor inputs at once.

Ian Goodfellow and Nicolas Papernot have a good blog on machine learning security issues. One relevant post on why this is such a hard problem:

http://www.cleverhans.io/security/privacy/ml/2017/02/15/why-...

Is it a big surprise that you can construct adversarial examples for algorithms? Don't humans have the same class of problems with optical illusions? And those are not even adversarial, just confusing.

If we constructed truly adversarial examples for human neurology, I bet they would be equally insane.

In some sense, yes, optical illusions are simlar to such "adversarial examples". But if you think about it, any kind of image is somewhat delusive, since we perceive it as whatever object it depicts while actually staring at a piece of paper with some ink on it.

Also, adversarial in this case seems to refer to images perceived differently by machines than by humans, so it's not really possible to create such ones for humans.

No, adversarial simply means deliberately trying to engineer false positives and negatives. This can be done against humans, machines, ants, trees, viruses ...

What is a false positive or false negative in this case? The "ground truth" here is what human perceive.

Optical illusion?

Wait... the dress was blue and black?

Camouflage is an adversarial example.

Thank you for your awesome response. It probably would have taken me hours of googling and reading to find these exact posts. A mere upvote is not enough of a thank you for such a quality reply.

Storrow Drive in Boston has several bridges over it like this. The worst part about it is that a car could be stopped under the bridge and it's hard to tell until you are on top of it (the car not the bridge). Granted the speed limit on the road is low enough that if you are obeying it you should be able to stop in time.

So a self driving vehicle would have to somehow know that the road dips and from that know that they won't run into the bridge and know that there may be cars hidden after the dip.

Edit: tried to get a good photo from Google maps but since the Google Street View Car camera is high up off the ground you can actually see more clearly what is going on with the bridge then you would be able to at street level so it's not a good example. I wonder what implications camera height has on the safety of self driving systems. Also, the bridges I have in mind seem to be on Soldier Field Road not Storrow (Storrow turns into Soldier Field and I haven't lived in Boston for years so forgive the mistake :))

I think this isn't well-applicable to machine learning because if there are more data sources to be considered (like other cars' behaviour in addition to just camera images), they'll be always considered anyway. For humans, it requires considerable additional effort to do so, but for machines it's fairly cheap, so why not do it anyway?

The uncertainity part should ideally, in my understanding, be represented by confidences returned by neural networks. They don't claim "there's a crossroad ahead, and not a tollgate" but rather "85% match for crossroad, 30% match for tollgate, [...]". If those results are not distinct enough, the surrounding application should probably go into a more cautious mode and slow the car down to begin with. I suppose that's what such systems do, but maybe someone with more field knowledge can confirm/negate that.

Reply on news.ycombinator.com