Hacker News Reader

Comment by njarboe | Hacker News Reader

And in the current climate of "all new things must be super extra safe to be used, even if it displaces much worse technology", I think machine driving cars will have a hard time being accepted. Imagine the autopilot is 100 times safer than human drivers. Full implementation would mean about a fatality a day in the US. It does not seem this would be acceptable by the public even though 30,000 annual human car fatalities could be avoided. Maybe we should use our education system to help heal us from the safety first (do nothing) culture we seems to gotten our selves into in the US.

This is a similar situation to nuclear power. With nuclear, all your waste can be kept and stored and nuclear has almost zero deaths per year. Contrast to the crazy abomination that is coal burning.

View on HN

Nuclear power plants, plane crashes, terrorism, school shootings, even some rare diseases (the mad cow disease led to the slaughter of millions of cattle, having killed in the UK less than 200 people cumulatively, whereas the flu can easily kill 10,000 people in a single year just in the UK).

I don’t think it has anything to do with technology and automation. People are just very bad at reasoning with probabilities. This is why lotteries are so popular. And it is fuelled by unscrupulous (and even less apt at grasping probabilities) media trying to make money out of sensionalist headlines. How many times a week am I told that x will cause the end of the world, replace x by flu pandemics, Ebola, global warming, cancer as a result of eating meat or using smartphones, etc.

Funny, Nassim Taleb, a worldwide (if controversial) risk probability expert, would say that you are the one which is being very bad at reasoning with probabilities, because you are putting Ebola and terrorism in the same class of risk as falling off a ladder:

https://www.youtube.com/watch?v=9dKiLclupUM

I would agree with flu (which is a fat tail risk, eg the spanish flu), but not terrorism (these are small local events) and ebola (ebola has a very low risk of transmission, it is just that it is very deadly once transmitted).

But classifying risks is one thing. Worrying in your daily life to the point of opposing a public policy is another. An asteroid wiping out life on earth is an even fatter tail risk, but the probability is so low that it is not worth worrying about. People have not rational reason to worry about ebola, terrorism, or plane crashes.

I think he/she meant something different: it's not about how big/small are the chances, but about how much we know on the probability distribution.

You assume you know the probability distributions, you compare them and determine that nuclear will kill less than coal burning.

Except that you might have never observed a black swan event before (eg. massive sabotage of nuclear plants).

So the probability distributions you inferred from past events might be wrong.

The case of a lottery is different as the probability distribution of outcomes can be determined with certainty.

Does it currently really reduce fatalities if you control for car cost, age group and area driven? I‘m unconvinced of the current statistics.

I think you could have just stopped that sentence after does it currently really reduce fatalities?.

why? got better data than me? please share.

Currently it is probably worse than the average driver. But here is a trolley problem for you. What is the ratio of people killed who volunteer to test out an auto-drive to the number of lives saved by that early testing? 1:1, 1:10, 1:100, 1:1000, 1:1,000,000? A million people die in vehicle wrecks each year.

Those are clearly not the only two options. Waymo appears to be far and away the leader in this space, and they've killed zero people.

Yea, most self driving miles are simulated, not done in meat space.

5 million are in meat space, too few to derive anything about safety since on average a fatal crash happens every 100Mmiles

Can't find how many of these miles were driven on roads with a posted limit above 35mph

The problem is that it's not just the drivers who are the volunteers getting killed. When you kill pedestrians as Uber did, they did not volunteer to be killed to test your software.

Yes automation is good, but Tesla is being needlessly reckless. They can easily be much more strict with the settings for auto-pilot but they're not.

If you are talking about the world a large part of that million are killed on the roads where no self-driving car would be able to drive anyway. Also "normal" drivers do not usually get killed in crashes—unless it involves reckless driving. So probably self-driving is safer than drunk driving, reckless driving or driving on the roads with the extremely poor infrastructure, but compared to responsible driving? Also, why is this so black-and white? The safest current option is technology assisted driving, but no much talk about it.

I'm just throwing something at the wall here: Just start with a purely assistive package like most big manufacturers have. Let buyers opt-in to a test program (giving them, say, a 3% reduction on the list price), where all their inputs and outputs are recorded and sent to the manufacturer. Manufacturer can now assimilate that data into a model of car and road and test their fully autonomous software on that model and search for situations where the software gave much different input compared to the human driver. Check these situations in an automated way and/or manually to see whether it's a software error or improvement. IMO this approach would have had a high chance to find the bugs that caused the two recent fatalities, as I'm convinced that most human drivers would have done better there.

After years of doing this (which seems to be close to what Waymo is doing if I understand correctly), the autonmous software should be way better than what is being pushed out now by Tesla and Uber (and probably a bunch of others).

Sort of assuming Tesla is doing this already.

If they do I'm surprised of the latest accident tbh.

I'm not reading a position in what you've said, but there is an observation here to draw out for the crowd. It doesn't matter all that much if after controlling for those variables it is relatively less safe; in absolute terms it just needs to be as safe as the worst human drivers.

That is, we have an accepted safety standard - by definition, the worst human driver with a license. If a Tesla is safer than that, the rest is theoretically preferences and economics.

I'm not saying that the regulators or consumers will accept that logic - if airlines are any example to go buy, they'll take the opportunity to push standards higher - but I think the point is interesting and important. It is easy to smother progress if we don't acknowledge that the world is a dangerous place and everything has elements of risk.

I don't agree with this position. Autopilot is a new thing that governments need to decide whether it's allowed or not (since it is operating dangerous equipment, so the state has an interest in doing the right thing). If improved safety is not a demonstrably valid argument for allowing it, it IMO doesn't pass the bar for what should be allowed. At that point it's purely a money-grabber from corporations that want the cookie (first to market with fully autonomous vehicles) without paying for it (heavy investment in testing).

What about if it could pass a driving test?

Since it would be one software being used many times over, I'd be fine if it could pass a large number of driving test with very low failure rate, considering driving tests are quite random in how they are applied. The question is however which driving test. My experience in Switzerland was that the practical driving test was quite hard to pass, I had to train a lot to satisfy them completely and they route you for one hour quite randomly around a European medieval town and its outskirts, i.e. a place that's quite hard to navigate correctly as it's not built for cars.

Considering that tesla w/ autopilot enabled is less safe than tesla w/ autopilot disabled, I'm going to go with "no, if you control for those factors it doesn't reduce fatalities".

and how did you reach that conclusion?

Oh is that the case? How long is this known? In that case I find it should be taken off the market by the government.

Source?

Anyone driving with it for an hour will probably encounter a situation they have to take over. Seems pretty obvious. It's a cool tech demo with bugs that can kill you.

That proves that a Tesla is dangerous if you enable the autopilot and then take a nap. It doesn't demonstrate that it's more dangerous if used as intended.

I haven't ever seen it save anyone, only introduce more problems for the inattentive driver.

Biggest factor to control for is drunk driving, which is ~30% of all deaths IIRC.

Good point. I wonder whether the ratio of drunks behind autopilot wheels is different than average. Could be lower (because richer) or higher (because richer).

There could easily be more than just a wealth differential, e.g. people who are more are of their emissions impact might also be be more aware of intoxication impact.

I'm any case, "statistically more safe" is a weak argument, e.g. we would be terrified of boarding a plane if they were merely statistically more safe than driving (by a small margin).

>e.g. we would be terrified of boarding a plane if they were merely statistically more safe than driving

Don't know why you're treating that as a hypothetical. Many people are terrified of flying and it is safer than driving.

What that commenter is getting at is that the term "statistically more safe" is meaningless because it can be such a wide net of meaning, e.g. a very small margin (1%) or a large margin (50%).

It's 28% in 2016 according to NHTSA [0], 10,497 deaths.

Certainly auto-pilot is worth it if the driver is drunk. I wish that all cars were fitted with alcohol measuring devices so that the cars won't start if you're over the limit.

Drink driving is still a massive problem in Belgium where I live. Although you get severely fined (thousands of euros) it's up to a judge to decide if you should have your licence taken away [1]. Typically you have to sit by the side of the road for a few hours then you're allowed to continue. In the UK it is a minimum 1 year ban.

[0]: https://www.nhtsa.gov/risky-driving/drunk-driving

[1]: http://archive.etsc.eu/documents/C.pdf

> Similar situation to nuclear power.

I think the issue with nuclear power is more the size of the blast radius. For example, the U.S. population within 50 miles of the Indian Point Energy Center is >17,000,000 including New York City [1]. The blast radius of a self-driving car is a few dozen people at best. It's not evident that nuclear technology is uniformly better than solar or wind, but it's probably expected-value positive.

[1] https://en.wikipedia.org/wiki/Limerick_Generating_Station

> And in the current climate of "all new things must be super extra safe to be used, even if it displaces much worse technology", I think machine driving cars will have a hard time being accepted.

Exactly this. People will not accept being hurt in accidents involving AI, but will accept being in accidents caused by human error. There is no way this will change, and car manufacturers should also have realized this by now.

As far as I can see, the only viable solution to the "not quite an autopilot" problem that tesla (and maybe others) has created for themselves is this: Just make the car sound an alarm and slow down as soon as the driver takes his eyes off the road or his hands off the wheel. The first car that doesn't do this should be one that is so good it doesn't have a driver fallback requirement at all - which I think is two decades out.

> It does not seem this would be acceptable by the public even though 30,000 annual human car fatalities could be avoided.

The flawed assumption behind this thinking is that all accidents have "accidental" factors. Alcohol and drugs and the poor decisions of young men are _huge_ components of that statistic. You also have to consider that motorcycles make up a not insignificant portion (12%) of those numbers.

It also completely ignores the pedestrian component (16%) which is due to a huge number of factors, of which, driver attentiveness is only one. So many pedestrians die, at night, on the sides class B highways that it's silly.

EDIT: To add, Texas has more fatalities on the roadway than California. Not per capita, _total_. This is clearly a multi-component problem.

And you also have older cars with less safety features, not sure how is in US but in other countries you can stil see old cars that are missing airbags.

So in fatalities statistics the fair thing is to compare Tesla with similar cars.

That’s a good point. Not just airbags. It starts to be common to have cars that apply the brakes automatically when they anticipate a collision. I think some research has been done on cars being designed to do less damage to a pedestrian in a collision, I don’t know if it has lead to design changes. Self driving cars may end up being only marginally safer than other new cars.

But safety is not in my opinion the main rationale for self driving cars. Convenience and maximisation of traffic are.

>>Convenience and maximisation of traffic are

I think the convenience will mean more traffic since many who now use public transport will use their self driving car instead. So in total autopilots could be worse for the environment and require even more roads

I agree with the traffic increasing, though as we are moving toward electric cars, not sure this is an environment problem. At least not an environment problem in big cities.

Electric cars still impose a huge environmental cost since they need to be manufactured in the first place. And the batteries need rare earths which are, well, rare.

While there is ongoing research into novel types of batteries made from more common materials, I wouldn't be surprised if the next war is about lithium instead of oil.

Agree, I am not convinced it is a net benefit for the overall environment. Plus they will consume more energy because of inefficiencies in transport and storage.

But there is still an immense environmental benefit: they will pollute in places (industrial areas, mines) that are not places where people live (big cities). So the population will benefit a lot from moving where the pollution takes place. And it's not just air. Noise pollution, dirty buildings, etc.

> Plus they will consume more energy because of inefficiencies in transport and storage.

Is that so? The larger engines in fossil power plants are more efficient than a fleet of small ICEs, but I don't know how this added efficiency compares against those inefficiencies along the way that you mentioned (and of course, a complete picture also needs to take into account the energy cost of distributing gas to cars).

> I think some research has been done on cars being designed to do less damage to a pedestrian in a collision

European safety tests rank pedestrian protection in addition to passenger protection https://www.euroncap.com/en/vehicle-safety/the-ratings-expla...

Outfitted Volvos will actually lift the hood in an impact to protect pedestrians https://support.volvocars.com/uk/cars/Pages/owners-manual.as...

If we do not push for safety then the manufacturers would invent some bad statistics and cheap out on sensors and engineering after they pass the bad statistics percentage.

The traffic issue it won't be actually solved with this kind of self driving cars, you would need new infrastructure, like modern metros/trains.

The dream of fast moving self driving cars, like a sword, I don't think is possible without a huge infrastructure change and I think the sword of cars would need a central point of control.

You will increase traffic without material change to infrastructures if you have no car parked in the street (particularly in Europe where streets are typically narrow) and you have better traffic flow management (coordinating self driving cars), variable speed limits, etc.

What is the minimum time you guess it would take to have a city with only self driven cars that could just drive by following lines and other specific signals?

I do not think this will happen in an existing city for at least 20 years.

I can imagine scenarios where a glitch or something else could cause tons of traffic issues in a city with only self driving cars.

I am not against self driving cars, I don't like the fact that this startups arrived and push alpha quality stuff on the public roads, it will create a bad image for the entire field.

If all newly produced cars are self-driving cars in 10 years, I think it is reasonable to ban non self-driving cars 10 years after that. So 20 years sounds about right. It's not that long for such a fundamental change.

And yes it will create its own problems. Like if these cars get hacked, they will cause serious damages.

Or maybe having a population with no hard challenges in their youth, like learning to drive and surviving. Seems to lead to people incapable of handling adulthood or developing judgement about risk.

So if you think the timeline is 20 years at best then we need that present self driving cars be able to drive on present roads with human driven cars, pedestrians, bikes, holes in the road, some bad marked roads, roads without markings, roads with a bit of snow, heavy rain,fog.

Adding some extra markings, special electronic markings for this cars is not a solution, and in this 20 years since the traffic is mixed you can't consider lifting the speed limits or changing the intersection rules, so I don't see how it helps the traffic (except maybe on small streets if you consider that people won't want to own their car and will use any random car. this random cars should be super clean and cheap to make people not own their own)

> current climate of "all new things must be super extra safe to be used, even if it displaces much worse technology"

The "current climate" is a backlash to the cavalier "move fast and break things" externality-disregarding culture of the last 10 years or so. We should be extremely conservative when it comes to tools that can potentially maim or kill other people. Not valuing human life should be an aberration.

> Imagine the autopilot is 100 times safer than human drivers.

Is it? Can it be proved controlling for all variables?

No it is not. But might be someday. Will that be good enough? My dad would spend 100k on a car that was 10 times less safe, if it let him keep using a car. I'm sure others would as well.

> No it is not. But might be someday. Will that be good enough?

I'm cynical enough to want to wait for it to actually be that good before bestowing the marketing goodwill on the company.

"Might be someday" doesn't really cut it for me any more.

Let's wait until we get there, then evaluate the situation.

The proper way to prove "it's 100 times more safe" isn't to let it cause some number of deaths and then go "welp, we tried our best but we were wrong, turns out it's less safe. Shucks". But that's exactly what Tesla, Uber, etc. all seem to be doing. "We'll compare our statistics once the death tallies are in and we'll see which is safer".

The most we have to go on for a rough approximation of safety is the nebulous and ill-defined "disengagements" in the public CA reports. From what I can tell, there's no strong algorithmic or safety analysis of these self-driving systems at all.

The climate about these things is sour because the self-driving car technology companies seem to want to spin the narrative and blame anybody but themselves for the deaths they were causing, and just praying they'll cause less of them once this tech goes global.

> the nebulous and ill-defined "disengagements" in the public CA reports.

The usual description of disengagements: "Disengage for a perception discrepancy" or "Precautionary takeover to address perception".

Except when referring to others, then its plain English: "Disengage for a recklessly behaving road user" or "Other road user behaving poorly".

For clarity on this point, "disengagement" has a specific meaning to the California DMV[0]:

> For the purposes of this section, “disengagement” means a deactivation of the autonomous mode when a failure of the autonomous technology is detected or when the safe operation of the vehicle requires that the autonomous vehicle test driver disengage the autonomous mode and take immediate manual control of the vehicle.

However, some self-driving car manufacturers have been testing the rules quite a lot by choosing which disengagements to report[1]. Waymo reportedly "runs a simulation" to figure out whether to include the disengagement in its report, but there's no mention of what the simulation is or how it might fail in similar ways the technology inside the car did! Thus, the numbers in the reports are likely deflated from being actually every single disengagement.

[0] https://www.dmv.ca.gov/portal/wcm/connect/d48f347b-8815-458e... [1] https://spectrum.ieee.org/cars-that-think/transportation/sel...

And even this pathetic and tootheless regulation was enough to drive Uber from California for a while.

With all the games they play with the numbers, Waymo still reports 63 safety-related disengagements for a mere 352,000 miles. This doesn't sound like an acceptable level of safety.

The surprising part is that it appears that Waymo is planning to start deploying their system in 2018. How can they even consider it with this amount of disengagements?

Your dad would be happy to get a car that was 10X less safe, but the rest of us on the road will be less so.

These are public roads, driving a dangerous cars put other people at risk.

What fraction of current drivers are 10x less safe? Probably all 16-20 year olds.

Probably around 3-4x[1], but point taken.

Inexperienced drivers cannot be avoided, but many states try mitigate the risk by limiting teenage drivers.

But my point is that whether to allow a vehicle that is not safer than a reasonable human driver should be not left to the car owner alone - there are other stakeholders whose interests must be taken into account.

[1] http://www.iihs.org/iihs/topics/t/teenagers/fatalityfacts/te...

You’re arguing from a horrible stance. Autopilot isn’t 100 safer than human drivers that is why people are concerned about safety.

It’s like if you said “I don’t get why people are concerned with every kid bringing a gun to school, if that would make the schools 100 times safer then shouldn’t we do it? Even if there’s an interim “learning period” where they are much less safe, doesn’t the end justify the means?”

I really can’t fathom the mindset of people who honestly believe that the only way forwards with self-driving tech is to put it into the market prematurely and kill people. In 10-15 years we’ll have safe robust systems in either case, let’s not be lenient with companies who kill costumers by trying to move to fast and “win the race”. Arguing from statistics about deaths without taking responsibilities into account is absurd. Would it be ok for a cigarette company to sell a product that eliminated cancer risks but randomly killed costumers as long as they do the math and show that the total death count decreases? Hell no.

> Imagine the autopilot is 100 times safer than human drivers. Full implementation would mean about a fatality a day in the US. It does not seem this would be acceptable by the public even though 30,000 annual human car fatalities could be avoided.

Your opinion will be irrelevant--the insurance companies get the final say. When that safety point arrives, your insurance rates will change to make manual driving unaffordable.

My hope is that the testing to obtain a driver's license becomes much harder to get and easy to lose. Like a airplane pilots license.

>your insurance rates will change to make manual driving unaffordable

I see people saying that. Why would insurance be more expensive than today--implying that manual driving becomes more dangerous once self-driving cars become common? It's already a common pattern that assistive safety (and anti-theft) systems tend to carry an insurance discount versus not having them.

Because the "manual" driver will almost always be at fault and the automated drivers will have the telemetry to prove it. For example, I recently got hit while backing out of my parking space--I'm legally at fault because of how the collision occurred. The fact that the other driver had tremendously excessive speed (a center mass hit on a Prius that literally lifted it onto the SUV sitting next to it) wasn't proveable by me--a self-driving car will not have that problem.

This means that the insurance companies with more "manual" drivers will be paying out more often and will adjust the insurance rates to compensate.

Reply on news.ycombinator.com