Hacker News Reader

“Autonomous miles without accident” is not the only metric. What matters is also how the algorithm fails.

To give an example. If you have autonomous miles without accident that is two times better than humans, but every accident tends to involve dark-skinned children and women, then the algorithm can have better statistical numbers in safety but still be rejected by humans.

Safety is a high dimensional problem that cannot be collapsed into one metric.

So if there's fewer deaths overall, including among black kids, but all the deaths are black kids that's not ok? I get your larger point but this one seems like a really poorly reasoned example.

It would depend on the details.

If black kids are dying because the car accelerates in school zones and swerves directly into black kids at high speeds, you can imagine that would obviously be not okay no matter what the statistics are.

My point is that a single metric like “deaths overall” is insufficient. The failure modes need to be reasonable and sympathetic to a human.

The premise of GP may have been slightly different. If a safety improvement reduces child pedestrian deaths by 100% in groups A, B, C, and D and by 50% in group E, that safety improvement should be welcomed, even though all child pedestrian deaths are now in group E (even though that group is also better off).

That is completely reasonable and would be completely unacceptable politically.

Perhaps it could be improved by being explicitly programmed to kill 25% as many children in groups A-D vs the original baseline. Then let activists argue why that figure should be doubled.

Computer vision systems often fail to identify people with darker skin.

That's what "equality of outcome" is; it's not about better it's about equality. When equality of outcome is implemented by raising everyone to a better outcome then it's a fine thing but that's not inherent to the underlying concept.

Testing shows that Teslas specifically have trouble identifying young (short) children as people to be avoided[0]. To your point, if Tesla were able to provide in the future that they actually do result in fewer deaths, but that the deaths which remain were heavily concentrated in young children, I think people would have a serious problem with that, regardless of whether the overall count of children dying was higher or lower. It's irrational, but that's how humans are.

0. https://dawnproject.com/wp-content/uploads/2022/08/The_Dawn_...

As you may have unintentionally alluded too with your reference to Concorde, there is no hard number or line. It's purely a public perception thing. There is no definable threshold that is "good enough".

The tech needs to stick around long enough and proliferate to a large enough cross section of society to become mundane (something Concorde failed to do). The people who are not satisfied will never be satisfied by an improvement in the numbers. What will change their minds is when it becomes clear that nobody else cares and they're just yelling at clouds. Realistically what that means is a generation has to come of age after the tech has proliferated and accept it by default and don't see it as pushing the limits of what's reasonable. From there the acceptance gets back-ported.

Serious question: will we demand perfection from autonomous cars before we allow them out of “test” status?

I don't think anyone is demanding perfection, but they are quite reasonably demanding that autonomous driving doesn't cause any accidents that wouldn't have happened if a human was driving (not that this is clear from the accident in the article).

The bar issimply 'better than a person'. If a car apparently changes lane and drives into another car without the cause being clear it's not there yet.

> quite reasonably demanding that autonomous driving doesn't cause any accidents that wouldn't have happened if a human was driving

That's totally unreasonable. If a self-driving car prevents 10,000 accidents that human drivers would have caused, but causes 20 accidents that human drivers would have prevented, we should accept it without demanding further safety improvements first.

> The bar issimply 'better than a person'.

This, on the other hand, is totally reasonable.

They need to be significantly safer. Not because humans are a particularly great, but as a human I understand the psychology of other drivers. Faced with another driver I know I can usually predict better what their failure modes are and compensate accordingly. This goes completely out of the window with self driving cars, whether I'm driving and must deal with them or in situations where all the cars are autonomous. And, particularly this is all bad when being a pedestrian.

Truthfully I despise the way this is going. Why can't we just massively reduce car use?

In my mind, I would be happy with 75% of the risk of human driving.

That is enough to be clearly better and clearly safer. It is a big enough margin to prevent lying with statistics ('oh, we only count highway miles', or 'yeah we were including with that statistic the young drink drivers').

It also isn't too unrealistic. If (to simplify things) you imagine that every accident has two cars involved and is the fault of just one of them, then a perfect self driving car that makes no mistakes will still be involved in half as many crashes (ie. 50% of the risk). I therefore don't believe it to be possible to exceed that while most cars are still driven by humans.

I would be completely happy with my government telling car companies they can do whatever they like as long as their accident rate per passenger mile remains under 75% of the nationwide average. I'd be happy with that instead of car standards too - if your manufacturer has a safety record exceeding that, they no longer need to do crash tests, brake performance tests, meet headlight regulations, whatever.

How is it lying with statistics to take into consideration the prevalence of drunk driving?

When you get in a self driving car you want to be confident you are safer in that car than in one you drive yourself.

If you are an honest citizen and don't drink drive, then you want to be comparing crash rates of other honest non-drink-driving citizens.

Alice says "I never drive drunk, so the accident bar for self-driving cars to clear shouldn't take accidents caused by drunk driving into account." Bob says "I never use my phone while I'm driving, so the accident bar for self-driving cars to clear shouldn't take accidents caused by phone use into account." Carol says "I never drive exhausted, so the accident bar for self-driving cars to clear shouldn't take accidents caused by exhausted driving into account." And so on, since for every bad driving behavior, there's probably at least one person who's never done it. If they all get what they want, then self-driving cars will have to be perfect before they're allowed, even though they could save a lot of lives by going into service long before they're perfect.

> will we demand perfection from autonomous cars before we allow them out of “test” status?

Not perfection, just no human injuries for the limited duration of the tests, that is, for a few billion km or so.

That's a standard no human driver could possibly achieve, for the simple fact that no human drove a billion miles. I think setting such standards, which will be close to impossible to achieve (and/or take centuries) is helpful.

How many human injuries are there per few billion km when humans are driving? Why isn't that the baseline we compare against?

On highways, the number of human injuries is a lot lower than you think. Most accidents are on surface streets, which is not where most "self-driving" features are purported to be usable.

> On highways, the number of human injuries is a lot lower than you think.

Sure, but it's not zero.

In the US in 2021[0], 5658 people died on rural and urban interstates combined. This was 13.2% of all roadway deaths.

0. https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...

The reason why I'm against self driving cars is simply the fact that they're cars at all. I sincerely have no idea why we are collectively spending so much time and resources to avoid the solution we've had for years: public transportation. I would much rather live in a world where I have to deal with the minor inconvenience that is waiting 20 minutes for a train to arrive but get to my destination cheaply and easily than all the insane beaurocracy and overhead that comes with cars existing at all.

> I would much rather live in a world where I have to deal with the minor inconvenience that is waiting 20 minutes for a train to arrive but get to my destination cheaply and easily

You can already live in such a world

If you cannot understand the position of your opponent, what hope do you have of a productive debate where we end up with a better outcome?

Self driving cars could be the public transportation of the future. Like taxis with the fraction of the price. It would be something that would dramatically reduce the need for private cars in sparsely populated areas, and even reduce the need for short and medium length flights.

I love public transportation, but the possibility of low cost taxis is very exciting.

I'm much happier to put up with the expense, bureaucracy, and overhead of owning a car than I would be if every trip, every day of my life took 20 minutes longer (and for a lot of places I go, 20 minutes is an underestimate). If you'd choose the opposite, that's fine, but don't do things like banning cars that would restrict my choice. That's just as unfair as banning public transit would be.

We're in agreement then. My ideal world is one where we have cities designed for public transportation (including walking), which would eliminate or at the very least reduce the current problems those methods of transportation currently have for everyone, while the current system provides somewhat efficient transportation but only for those who can afford it. I think spending money and human resources into improving cars instead of public transportation just exacerbates that problem, so it just frustrates me on a personal level to see that this is the direction we're headed to.

If you like public transportation, then I’m delighted. I hate it and don’t want it forced on me. I think that ultimately autonomous cars are likely to replace much of public transportation, or at least be a large part of such a system.

The problem is we're currently living in the opposite situation, one which is so normalized most people don't even realize it. We have entire cities built around and for cars instead of people, with public transportation treated as an afterthought and just walking to places not even considered as a possibility. I wouldn't want to force public transportation on people either, but pretending the alternative of forcing cars on everyone else is better, desirable or not current isn't productive.

As a minimum it must not switch lanes, because this is where the dangerous accidents happen, and you cannot fix this as the other driver.

Yes.

Robots killing people are much scarier than humans killing people.

Unfeeling monsters taking lives are much more frightening because we have zero mental models for what motivates them and therefore no way to predict or protect ourselves from them when we encounter them. Statistics don’t matter.

Psychopaths are scarier than normal people even when they have the same outcomes.

We’re just desensitised to motor vehicle accidents. Self driving car accidents are still novel. A million people killed worldwide per year in motor vehicle accidents is just a statistic.