Hacker News Reader

Comment by burntoutfire | Hacker News Reader

What about "practical error bounds", i.e. testing the system through millions/billions of miles driven?

Empirical evidence has shown that all Tesla cars have the same failure mode when a stationary obstacle is on the highway. This is not some fringe failure that happens because of wrong classification, it's how the Tesla Autopilot was designed to function. Tesla has claimed explicitly that it's the driver's responsibility to avoid such a situation.

Tesla is not a good representative of the field; they are waaaay behind Waymo (Google) and Cruise (GM).

Miles driven is a useless metric. Stick your vehicle on a treadmill and have it drive a billion miles. What does that tell you?

Edit to add: a better metric would be something like "billions of decisions made where human life was at stake".

It's pretty clear the they meant "distance driven in ordinary conditions where humans normally drive cars".

That isn't the case though. Gm and Google both tell you that their incidents per mile (I think they use 100000 miles or something) is constant, because as they get better they test in harder situations. A automatic car on a desert freeway (no traffic) is easy compared to freezing rain during the afternoon rush hour in Minneapolis. (afternoon implying most people were out and so the decision to stay home isn't available). That is just on hard situation that comes to mind, I think those doing self driving cars know of others.

This is how some hardware is already certified for use in safety critical systems. ISO 26262 calls it 'proven in use'.

> testing the system through millions/billions of miles driven?

Makes sense doesn't it?

Well, actually no. The Google cars drive the same route every day in Mountain View with no deviation, so those millions of miles are really the same 10 miles over and over. Even seen 3 in a row behind each other.

Fools the regulators, and apparently you, though.

Deaths are often measured in deaths per 100,000,000 miles driven, with a current measurement of 1.18 deaths per 100 million miles[1]. If a car is driving 60 mph 24/7, it takes 190 years for it to drive 100,000,000 miles. Let's assume that, in a decade, 1% of cars are on autopilot. There are 37,000 deaths per year in the USA due to car accidents[1]. Therefore, in order for Tesla/Google/GM to claim that it is equivalent to normal cars, it has to drive 370 * (100 million/1.18) ~= 31,300,000,000 miles, with only 370 accidents. If $COMPANY really had, say, 400 accidents, then it would have to make up the additional miles. In specific, it would have to make up 30 * (100 million/1.18) ~= 2,500,000,000. That means a fleet of (190 * 25) = 4,750 cars driving 24/7 for an entire year. That's not impossible for $COMPANY to do, but it is starting to become unreasonable. What's more likely is that $COMPANY just straight up lies about its incidence rate to regulators. But that's hard to do, considering it's going to be public if someone gets hit by a car.

I don't think its possible to fool the regulators, but even if it was possible there are dramatic consequences for lying. Volkswagon tricked regulators into thinking their car didn't emit as much as it really did in 2007-2015, and ended up having to pay 33.3 billion dollars in fines and returns[2]. Volkswagon stock went from a high of around 27 to 12. 5 years later, it's only recovered to around 16. The massive lawsuit that would be formed if Tesla's (or other automated car maker) car was found to be unsafe would likely be even larger. Furthermore, it would crush the self-driving car business semi-permanently. VW at least still gets to make environmentally safe cars, Tesla and Waymo's main value proposition is dead in the water.

Basically, even though there are incentives to lie and cheat, I think the incentives to try to make something that is safe is far, far greater. And while I'm sure there are Google cars driving the same route every day, I'm sure there are also tests being done in all sorts of conditions all over the planet. I think that practically, socially, and politically speaking, we can use billions of miles driven. The stakes are way too high to lie.

[1] https://en.wikipedia.org/wiki/List_of_self-driving_car_fatal... [2] https://en.wikipedia.org/wiki/Volkswagen_emissions_scandal

Reply on news.ycombinator.com