Hacker News Reader

Comment by wongarsu | Hacker News Reader

So uber fails to predict even completely linear paths traveled by objects if the system isn't trained on or expecting that specific type of object? That sounds like an even bigger issue than "we didn't think pedestrians could exist outside designated crossings"

View on HN

What's crazy to me is that an "Other" object isn't flagged for a complete stop or a major alert to the "safety" driver. This seems like a major case for when the software should stop being autonomous, when it isn't sure.

Likely the story is that their software flags a lot of things "Other" without figuring it out upon closer inspection.

For those who think "well, pedestrians don't belong on the road so this wasn't much of a bug" (because I'm sure there are some of those): just imagine the "other" was a large chunk of debris, or cargo fallen off a truck. The car would have crashed right into it.

There can be many obstacles on the road, stationary or non-stationary.

AI can get away with requiring humans to take control in emergency/unusual situations, but not when it is supposed to be "self-driving". The software at least needs to recognise problematic situations and request a safety override or come to a safe stop, it can't just shrug it off and move on.

Debris tends to be stationary most of the time. Sounds like the car could have handled that.

I agree with that ADS being shit. Objects, once identified, should be accounted for in later updates in some form. E.g. in the way of "pedestrian now behind truck next to me". How the hell can an object switch status back and forth, loosing all its history? This should make the system be extra cautious at the very least. But yeah, safety doesn't seem to have been that big of a priority.

Most of the time, yes. But if you've ever been in a mountainous region, you'll know that it won't be stationary all the time and that it's most dangerous when it's not.

There was an emergency braking feature that would engage when a collision was imminent. It fired 1.3 seconds before hitting Herzberg. But Uber had disabled that function because there were too many false positives, so the car just alerted the driver (who was not paying attention) instead. https://www.washingtonpost.com/news/dr-gridlock/wp/2018/05/2...

1.3 seconds is definitely not enough for a human to engage brain. I agree the human should have been paying attention and might have spotted the problem sooner, but that alert is completely useless.

Alerting a human (who wasn't even in the driver's feedback loop at the time) 1.3 seconds before the collision? That is equal to "hey, look at this unavoidable crash that we're driving into!"

It is ridiculous to expect that the driver can be alerted and react in any kind of reasonable time frame unless they were already paying attention, in which case what use is a self driving vehicle?

They are supposed to be paying attention in these self-driving car tests. If the driver was negligent then they should be held at least partially accountable.

FWIW, we now know Uber's technology didn't do this, but we don't necessarily know the nitty-gritty details of how other self-driving systems work.

For me, the most important take-away isn't something about Uber being sloppy. It's that the distinction between "machine learning" and "artificial intelligence" really matters, and the efforts of marketing teams to over-hype ML by selling it as AI may be hindering our society's ability to make cogent public policy decisions around these technologies.

(No, I'm not really interested in haggling over ideas like "hard AI" and "soft AI" here, either. Normal people don't split those hairs. Nerds generally don't, either, except when they're trying to score easy points in a debate.)

The machine learning part could have gone better: it didn't expect a pedestrian and jumped between vehicle, other, bicylce and unknown. But that would have been good enough, the failure is in the code around that. When your classification is unstable or low confidence you can't trust it, which in a safety critical system would mean falling back to something else. That could be using a different classifier (e.g. linear interpolation of the object's position to find out if we come anywhere close to it) or just giving up and stopping the vehicle. Just continuing as if the fluctuating results were reliable wouldn't be acceptable in any industry.

What kind of tech do you expect them to use? While I hope it wasn't, I'd not be surprised with them just running a classifier over the current scan, have it spit out "vehicle" at coordinate A, search the previous scan's analysis for a matching vehicle close by. How do you handle uncertain/flip classifications? Or objects appearing to suddenly "spawn"/"despawn"? A proper "what object was that in the past" would likely require its own AI. Reliable linear interpolation doesn't seem that simple to me.

From what I remember, the pedestrian was detected early enough but classification jumped between two or three classes and it never got a prediction vector, basically being a standstill object (at different locations) that was never predicted to cross the trajectory of the car.

Reply on news.ycombinator.com