Hacker News Reader

Comment by TeMPOraL | Hacker News Reader

The thing is, human "machine vision" has graceful failure modes. You don't switch from seeing cars to seeing nothing to seeing an elephant just because the input got a little noisy. The same cannot be said about current ML demonstrations - because they operate on just vision. Humans continuously reconcile visual input with their model of the world and with other inputs, to the point of overriding visual data if needed.

I suppose you could make ML avoid sharp changes in recognition, but I don't trust the current neural-network models to do so reliably.

View on HN

Another issue is detecting issues and fixing them. For example, if dirt gets into my eyes while driving, I immediately know and do what is necessary to clean my eyes before continuing on my drive. On a car, what if there's a small spec of dust on a camera that blocks some part of the view. Is the car smart enough to figure out something is wrong and refuse to go?

The AP2 feature in the Model S currently alerts the user when vision is obstructed and refuses to go into autopilot mode. If you're in autopilot already it makes noise and flashes warnings to tell the driver to take over. I assume if the driver doesn't it will slow down and come to a stop, but the one time this happened to me I didn't give it a chance to do so!

You've never been temporary blinded by a low-sun? Never sneezed and had your eyes closed at the wrong moment?

This is the same thing..

Humans know when they can't see properly. The problem with adversarial examples is that the neural net thinks it's doing great. You have to manually specify every possible failure mode for the neural net before hand -- if you miss one, you might have an accident.

Usually. Drunk and sleepy drivers blur this distinction, but you still make a good point.

The real world is not filled with adversarial examples. Further, NN often does not think it's doing great when providing off the wall examples. Further this is a car, with GPS, where simply stopping is a completely viable option 99% of the time.

The truth is people really really stuck a driving, but it's easy enough that the vast majority of mistakes don't result in any problems.

>simply stopping is a completely viable option 99% of the time.

For a naive definition of "vaiable" that does not take into account other people that need to use the roadways. A car stopping or even slowing drastically in a lane will have the same negative affect on the transportation system as a minor accident.

These cars are unlikely to stay stopped for long as there are going to be passengers in them.

Now, we could get into hypothetical situations, but the reality is minor accidents are already really common like with so many other things machines can be better than people without being perfect.

I really want to see how cars deal with the massive code explosion as they have to account for things like tak tak gangs.

I posit now: "as the problems with automated cars become evident, solutions provided will tend towards Skynet as t increases."

You mean like we have every day already? Human drivers constantly make mistakes, and in many cases they aren't even aware of it or their affect on the drivers around them. Most don't cause an accident, but the behavior doesn't get corrected and that is what it eventually leads to.

Slowing for a turn then deciding to continue on, merging without checking blindspots, running red lights, speeding next to the short on-ramp lane, not following a yield to right turn sign, ignoring people in crosswalks, etc. All of these things happen all the time, I see at least one example every time I go out somewhere.

Aren't general adversarial networks designed to solve this?

> Humans know when they can't see properly

There's plenty of conditions that are more difficult to humans: cars that are painted with the same color of asphalt seem to disappear in highways. Also, fog.

Motorcyclists have a word for this phenomenon. It's called SMIDSY - "sorry mate, I didn't see you." Clear road, good vision, bike traveling in a straight line within the speed limit, car still somehow pulls out and hits the bike. Even worse, the times when the bike is stationary at a red light and the car just drives through it.

And it's not limited to cyclists. I think the general term used in cognitive science is "Looked but failed to see".

It's not. When you're being blinded by sun, you don't suddenly see elephants or spaceships or comic books. You also don't think the world just disappeared - you know to just ignore the visual input.

Also, I'm more worried about temporary, unexpected configuration of things that suddenly make a NN see something weird.

That doesn't sound as an insurmountable problem.

Humans don't see elephants suddenly because a prior is telling them that an elephant is improbable in that context.

Something like that in a computer system shouldn't be above the current capabilities.

I agree it shouldn't be above current capabilities. But I'm yet to see anyone doing something like this. And I'm definitely wary about taking the results of toying with "deep learning" and saying we've can put that on a car and expect good results - which is an impression I sometimes get in articles about those topics.

Also, this will stop working the day an elephant escapes from a ZOO and manages to get onto a highway. All the autopilots will declare the elephant's presence as impossible, and will happily crash into it.

Not likely. Computers can do a double-take, just like humans.

I'm not sure the teslas are going to be trained to see elephants either!

You don't necessarily have to train it to see elephants, but Neural Networks fail in pretty interesting ways. This that are very obvious to you and I have the ability to confuse a CNN[1].

[1] - http://www.evolvingai.org/fooling

Those aren't too bad (out of context of the Tesla article), I can see how the ai might misinterpret those images. The bagel one, for example, is pretty on point.

Those examples show that ML algorithms "see" things much differently from us. It also shows they're very vulnerable to nonsense data. How can I be reasonably sure that some random play of shadows and lights, that would never fool a human, will cause a failure mode in a neural network in a vision-only self-driving car?

Or worse - paint jobs created specifically to muck with image recognition.

Or when the graffiti becomes intentionally malicious (thinking of a variation of Wiley E. Coyote's tricks with paint and solid walls)?

Would these be the same graffiti artists that are currently maliciously painting over road warning and direction signs to fool human drivers? Say, whiting out a 'low bridge ahead' sign, so that trucks and double-decker buses crash into it...? Because, nobody is doing that now, and if they tried it, they would be prosecuted, probably for murder or attempted murder.

I don't understand where some people get the belief that automated vehicles will suddenly become targets of murderous or sociopathic pranksters?

Last year, there was a popular hoax about a graffiti of a Road Runner tunnel: http://www.snopes.com/road-runner-tunnel-crash-rumor/

> I don't understand where some people get the belief that automated vehicles will suddenly become targets

There is precedent when it comes to the automation of people's jobs (truckers, cab drivers, courier services) and fanatical condemnation of technological advancement; see everything from the Luddite's infamous wooden shoes to the spiking of trees to stop logging.

In the forests near my hometown, some environmental fanatics will string up wire across motorcycle and ATV tracks, or create blockages and attack riders with crowbars and pipes. I wish I were kidding.

As for current graffiti artists - they're taking down stop signs and road markers, defacing handicapped parking signs with stickers, tagging any and all road signs, and so forth. They're minor nuisances for human drivers, who can usually easily identify compensate for such changes. Computers, even those running neural networks, are not able to compensate so easily.

These look like a digital Rorschach test. With the minor difference that a human "knows" the image is not really an object and is being asked to interpret in some way; where the machine has no "knowledge" nor "understanding" but has an imperative to match this input to something in its repertoire of knowledge.

As a human, if you asked me to match these synthetic images to a most-likely real-world match, then my responses will also look "confused."

The thing is that computers also rate them with 99% certainty.

Julia Evans: How to trick a neural network into thinking a panda is a vulture

https://codewords.recurse.com/issues/five/why-do-neural-netw...

I guess the CNN was trained only with real images. They must add some [random noise], [crap], [glitch] and [collage] images to the training set. This may reduce the efficiency of the NN in the real world, but it will make more difficult to find funny examples of misclassification.

They will need to be if they are to be able to recognise all the potential real world phenomena they might encounter!

That's just Bayesian priors. Easy peasy for ML.

Haven't seen much Bayes in current deep learning trends though.

> You don't switch from seeing cars to seeing nothing to seeing an elephant just because the input got a little noisy

yeah, instead you get hallucinations, muscle twitches, seizures, mental breaks and many more ways a brain can fail to do it's job.

Sure, but a) we have thousands of years of experience with those, and b) none of those are caused by little random noise in the visual input. Brains are incredibly robust against noisy data.

Robustness is a key property of the methods and models used in self driving cars.

For a human, the noise isn't bad recognition, it's corrupted sensor data via distraction. It's one of the most common causes of accidents. For the AI agent an elephant will never appear unless it's been trained to be recognized. More likely it's recognized as an obstacle, which is ok. But keep in mind that years of Google's autonomous vehicle success make this pretty improbable, certainly less than in the average driver.

Much of the fear here is irrational and natural when humans lose control.

I'm not trying to express a fear that comes from humans losing control. I don't fear that - in fact, I can't wait until the day comes no humans are allowed to drive on public roads.

I was just answering about the concerns of self-driving systems relying solely (or primarily) on visual data from cameras. It's true this is how humans drive, but human visual system is much more complicated than what the current (published) state of the art in image processing seems to be. I do not trust visual-only systems today (especially if they employ deep learning shenanigans).

I know those problems aren't insurmountable, but I'd feel much safer if they threw in a LIDAR there too.

While I'm sure more sensors gives a better chance at consensus I don't think relying solely on visual data is a problem if done well (low false positive false negative rate, comprehensive trained, fault tolerant)

Our vision and persistent world model are good, but all humans are overconfident in their ability to drive in poor conditions (or even normal conditions, for many people.)

A computer isn't going to look down at a text message on their phone, fall asleep at the wheel, or get in a car while intoxicated. All which occur literally every day. People take driving for granted, because (at least in the US) we have to. Our sprawling suburbs couldn't function without it, so the barrier to entry is simply being a teenager.

Would I completely trust Tesla's auto-driving? No, not at all. But I at least trust them as much as I do an average driver, which is still just barely. The difference with Tesla's system is that one failure leads to improvement of the entire fleet, whereas one human failure maybe improves that one human driver. That means self-driving cars will continuously improve, while humans will just continue to be terrible drivers.

Yes! Tesla will take any death due to a failure of self-driving quite seriously indeed and work to fix the failure mode.

Whereas humans get the same kinds of fatal crashes over, and over, and over...

The Australian government has a "Black Spot" [1] program to provide federal funds (often millions) to fix up locations with a history of crashes. Bet it's cheaper to make those fixes to software.

[1] http://investment.infrastructure.gov.au/funding/blackspots/

Some adversarial methods work on black-box networks - has any work been done on applying those to human vision? We know there are a wide range of simple inputs that produce unexpected results (eg. optical illusions).

a) well, no. we only have about 100 years dealing with these problems in context of driving cars and we're really bad at it; b) that seems like shifting goalposts, what we care about is whether driver's picture of the world is correct and actions are sensible with respect to that picture, it just so happens that driverless cars' picture is visual (or close to visual), but if you're judging them on entirety of input and side-effects it's only fair to judge our brains by the same standard.

> You don't switch from seeing cars to seeing nothing to seeing an elephant just because the input got a little noisy.

You'd be surprised. All kinds of conditions can do really weird things to your vision. Migraine auras probably being the most common.

Reply on news.ycombinator.com