The only real solution is to get the false positive rate down. Which is why they still have human drivers until they do.
The only real solution is to get the false positive rate down. Which is why they still have human drivers until they do.
Yes, building a self-driving car is hard. But if you don't even have emergency braking working, then your technology is too underdeveloped to drive on public roads. Keep trying on the test track.
Personally, I doubt that they really considered that; I believe that their software was rushed, flawed, and totally not ready for real-world driving. But I don't think it's a certainty that false positive emergency braking is always better than running over obstacles, especially if "running over obstacles" is determined to be a sufficiently rare event.
It is entirely likely that Uber engineers evaluated this scenario, and (correctly) decided that it was safer overall to turn off reactionary breaking - and have that function performed by a human in the driver seat. If they hadn't, these cars might instead have been brake-checking people at 100x the rate.
It's easy to say things like "Uber should have waited until better safety features were available", such as the eye tracking suggestions mentioned elsewhere in this thread. But features like that take time - especially if they're development-only features that would have no place in the final product. Every additional safety feature pushes FSD deployment back.
Globally, 1.25 million people die from car accidents annually. That's over 3,000 people per day. For every day that you delay mass adoption of FSD, you are accruing massive amounts of fatalities that could have been avoided.
FSD does not have to be perfect, it's development will cost innocent lives - but if you're optimizing for minimal loss of life, it's the correct thing to do. Reactionary policies do the opposite of what you intend them to do - they cost more lives in the long run.
This accident alone demonstrates just how flawed the system is. This wasn't even an emergency situation to start.
These cars could absolutely pass that test.
This is literally how all human driving works right now.
Seriously, look around at other drivers while you're on a freeway or interstate sometime.
They're not driving, they're singing along with the radio, or shaving, or putting on makeup, or sending text messages.
They're not Luftwaffe aces with eagle eyes and steely nerves monitoring the fuel mix and oil pressure while scanning the skies for the silvery glint of the sun off the wings of a P-51 that may herald their last few moments on earth. They're bored people doing their boring commute and even when they have their hands on the wheel they're not really paying attention.
Yes, humans suck. But that's still orders of magnitudes better than uber.
If, given two scenarios:
1. We develop FSD very carefully. 0 lives are lost during FSD development. Ten million lives are lost by the time that FSD sees 100% adoption.
2. We develop FSD less carefully. 100,000 lives are lost during FSD development due to suboptimal performance before it is perfected. Five million lives are lost due to human drivers by the time FSD sees 100% adoption.
Would you really choose the former option? If your ethics board refuses option 2 in favor of option 1, it is they that are mistaken.
That is, without proof the new way will be safer in future by a specific proportion, how many excess deaths can the ethics committee sign off?
(You can make a similar, better founded argument for mass train transit. It is about 10 times safer than driving per passenger mile. So, reductions in safety that moves people from car commuting to train commuting (I imagine safety is expensive and cheaper transit would be more popular) might be worth it. No-one seems to advocate them though.)
How can you know the future in this case, I would not believe Tesla/Uber (the others) predictions , "trust us, let us kill 1 million people in the next 10 years but after that the number will go down to 100, trust us, we software developers are very reliable at predicting things and our mathematical skill are so great we can be sure even when we use NN that are unpredictable. And if we fail then the next startup will appear, promise the same thing and you have no choice then to give them also a check for 1 million lives to not discriminate between startups.
Its not a numbers game. Those are likely two different sets of people. You are saving some people by calculating in, that others get killed instead. It is different to for example, using an experimental treatment option on terminal patients. That would be, saving some of the dying patients but some might die regardless. The scenario is much more similar to the trolley problems cousin the transplant problem. Having someone else, uninvolved killed to save a larger group of people.
Seeing as the automatic emergency breaks were disabled, it seems the product wasnt good enough to detect false positives and was rushed by cutting the safety measure, accepting that putting it in live traffic the car would likely kill people unrelated to the research. It was a trade off between slower development and killing people.
I dont think you will find any ethics board that will green light you killing uninvolved people to save how ever many people. Of those i can think of who did that, quite a few ended up being wanted for crimes against humanity. We have ethics boards for a reason. Without them the ends can quickly justify the means.
Uber isn't a philanthropic research endeavour, and it can't rationalise killing people based on lives it couldn't or wouldn't have saved in the event of its technology actually working. The reason they put tech that reportedly hit things every 15k miles and had a near miss every 100 miles on the road as soon as possible has nothing to do with optimizing for minimal loss of life and everything to do with optimizing for a unicorn valuation.