It's grossly irresponsible of Tesla to ship a system that can't detect big, solid obstacles. This happens over and over. So far, they've hit a crossing semitrailer, a street sweeper, a car at the side of the road, a fire truck at the side of the road, and two road barriers. This has been going on for five years now, despite better radars and LIDARs.
The car in the video does seem to be braking strongly 1 second before the collision. There's a cloud emanating from the wheels.
> Humans often brake late, but seldom fail to brake at all.
Humans do fail to brake completely, e.g. when they fall asleep or have their eyes on their phone.
There are no brake lights on the Model 3. None at all. Based on this "reverse video", my conclusion is that the M3 doesn't even brake.
-------
Somewhat worrying is the pedestrian (the truck driver?) who jumps back to get out of the way of the M3. Not only did the M3 miss the truck, but the M3 didn't seem to see the pedestrian either.
Also note all the cars that followed and clearly hit their brakes. They're invisible or nearly-invisible, except when they go through the overpass's shadow.
Puffs past him and then straight into the side of the truck.
Tesla has been very stubborn about avoiding LIDAR and this is what happens as a consequence. The rest of the self-driving scene fundamentally relies on LIDAR to get things working. Believe me, if cameras were sufficient people wouldn't use LIDAR . LIDAR is expensive. I have spoken to quite a few people in the field and the lack of LIDAR in Tesla vehicles baffles them.
Radar is not a substitute for LIDAR. Sound is really high latency and has terrible resolution.
I would like to see how often things like these happen with Waymo.
https://spectrum.ieee.org/cars-that-think/transportation/sel...
https://www.wired.com/story/vw-audi-aid-luminar-lidar-self-d...
https://www.audiusa.com/technology/intelligence/adaptive-cru...
https://jalopnik.com/volvo-plans-to-be-the-first-to-include-...
https://www.nytimes.com/2020/01/22/technology/cruise-gm-self...
Doesn't look unsightly to me. Also given that every car company and self driving car uses LiDAR suggests that over time the costs will come due to economies of scale.
Also are people not forgetting that Tesla charges a massive premium for Autopilot ? People are willing to pay a lot for self driving more than enough to cover the cost of the LiDAR sensor.
This has been predicted for radar units over optical in other industries. It never happened. Optical became cheap and capable, radar stayed slow and expensive. I see no reason why this would change in car manufacturing.
I don't necessarily think LIDAR is a bad idea, but this logic seems kind of circular. Isn't the issue that nobody knows what's sufficient? At the very least the engineers at Tesla seem to think LIDAR isn't necessary (maybe not a slam-dunk argument but humans don't use LIDAR to drive, so it's at the very least plausible computer vision systems alone are sufficient)
A big flat surface that's not normal to you like the box truck may also not be a very large return.
Maybe in this particular instance you could look at the data and go "yep, that white surface is indeed closing in and not just some part of the sky". But how many false positives do you allow for "there's a huge thing in my way that requires emergency braking" in interstate scenarios?
It seems like the color of the truck and that of the road were near identical to each other. Maybe no different that what it looks like when the road changes textures to a worse surface or or hits an odd pot hole.
It is likely the car alerted the owner to take over, but did not brake on its own.
The thing about LIDAR is detecting objects becomes shit easy. One thing is for sure. I can guarantee such an accident would not have happened with LIDAR.
LIDAR isn't a surefire solve, it also has issues with large white surfaces and bright sunlight.
Making a computer vision system for a car that always knows what images mean the road is clear and what images mean the road is blocked with 100% accuracy is very hard.
That said, Tesla has also said they can determine the 3D geometry of the scene using only computer vision, which is certainly a known task (called photogrammetry or structure from motion). However it seems their algorithm failed in this case to accurately judge depth. With a clear picture and a well understood camera system it is pretty do-able, but I haven't looked at this article so perhaps the footage was not sufficiently clear.
Yes.
Unfortunately, this wasn't human vision: it was computer vision.
When both systems succeed, they by definition succeed similarly: I say by definition, because human vision is the only judge of computer vision's success.
When they fail, they fail completely differently.
It's a hard problem, but a system with more sensory inputs than the human optical system shouldn't solve the same problem worse -- this points to a real issue in the problem solving methodology, but doesn't do much to condemn the idea of camera/radar fusion.
Now, if the question was "Can Tesla do this reliably?" -- different answer.
I was driving on the highway the other day and there was a large black object ahead in my lane. I was about to take evasive action when the object shifted slightly. The motion was enough for my brain to sense it was a lightweight object (not a human fallen on the road) and a further movement in a puff of wind to establish it was an empty garbage bag rolling on the highway. No need for evasive action.
On the other hand if I had seen other debris/leaves tumbling in the wind but the black object was stationary my brain would have sensed it as a potentially heavy object and taken evasive action.
I remain skeptical that we can if-then-else engineer our way to self-driving until cars have the common sense adaptive behavior that humans have.
The first principles argument can be made just as well about setting up two cameras and some robotic hands to converse on HN and acquire karma.
Do you think the car understands cause and effect? Do you think the car actually groks object persistence? Do you think that the car has any sense of value attached to it's own existence? Or an instinct to increase the amount of time it has to resolve a reading it doesn't understand?
This is the failing of your lab trained neural network. It is only trained and proficient in what it knows, and only that. It doesn't have a sense of self. It doesn't have intuition, instinct for survival, or any of the millions of other simultaneous neural subprocesses that even the simplest biological creatures possess to keep themselves alive. It can't forget, neither can it learn in it's runtime state. It's a crapshoot whether it'll do everything it is supposed to how it is supposed to just like it is when a human (coincidently, another neural network driven creature) gets behind the wheel.
You won't build a perfect neural network for X just like you can't make a perfect human by selective lobotomy. There is only so much neural ductility that can be accommodated.
Really? My adaptive cruise on my bog-standard GM Volt seems to have no problem seeing static obstacles with it's "primitive" radar.
My vehicle is doing adaptive cruise to the car in front of it--including on banked curves and in Los Angeles traffic--sometimes dense-sometimes empty. This means that it is working out which car is in my lane--probably via beam steering and corresponding to my steering wheel position.
This doesn't seem to be an impossible problem. This seems to be a Tesla problem.
This one discusses that the Volvo system has the same limitation, and implies that most others do too https://www.wired.com/story/tesla-autopilot-why-crash-radar/
It's not the first case I've seen where Tesla's Autopilot seems to override AEB[1]. Definitely confirmed behaviour to-date and seen as early as Q1 of this year. Any car from 2009 or later with AEB would have avoided this collision.
It probably would be great if car software didn't follow the "move fast and break things" mantra.
---
[1] https://www.consumerreports.org/car-safety/automatic-emergen...
Looking at the video, a bunch of dust seems to be kicked up around 1.5 seconds before the impact, suggesting hard braking. Clearly it activated too late to avoid the collision, but it certainly would have reduced the energy of the impact, and thus potential injury to occupants.
That doesn't protect against operator errors of various kinds (set course into a mountain, didn't realize the autopilot was disengaged, ...), but is a large difference from streets. Even in planes pilots taking over when automation encounters a situation it can't handle is a risk due to potential confusion about the current situation, and on streets the reaction times necessarily to react to an autopilot failure are typically way shorter.
“Autopilot enables your car to steer, accelerate and brake automatically within its lane“
to me, seems to apply to this event.
To me, it also looks as if it almost hit a person, too (the driver of the van?)
You’re right, I hadn’t noticed that! That’s even worse. Seeing a person on the side of the road is not that uncommon and is a near-certain fatality if a collision occurs.
Systems exist which will override auto-pilot in the event of such happenings, like Auto-GCAS (ground collision avoidance systems) and equivalents, but these aren't really designed for sudden obstructions -- they are mostly designed to help regain level flight if a pilot becomes unresponsive during maneuvers and is pointed in A Bad Way -- but auto-pilot is not going to be engaged during such flight, anyway. But systems like this do have authority to disengage whatever other systems that may interfere with their operation, like auto-pilot.
There are many onboard systems to allow awareness of such conditions that will result in collision, but not much in control of actual flight characteristics on any but the most advanced or experimental planes.
I don't think civilian air planes have any collision avoidance radar, because as late as 2006 we get mid air collisions like this: https://en.wikipedia.org/wiki/Gol_Transportes_A%C3%A9reos_Fl...
In that accident the smaller plane had its transponder switched off.
A passenger on the smaller plane (which had zero casualties) wrote about it from his point of view: https://www.nytimes.com/2006/10/03/business/03road.html
There may be other sensors onboard that help with detections (it is a lot more expensive after-all) but the autopilot control system does no such thing.
Even with all the sensors onboard, the 747 would most likely run into something without assistance, in fact I've seen a few cases of this on Air Disasters.
We just put another driver in there to check things.
In the same way, what I think is:
driver - pretty safe
autopilot - pretty safe
driver + autopilot - safest
My point is that while driver+autopilot is claimed to be the safest option, I contend that it is less safe than driver only. At least at the current state of technology, and even including teens.
And in that time, how many people have crashed into other people, whether they're in cars or not? Buildings? Walls? Children?
Are you claiming that Tesla's safety performance is worse than that of people?
Tesla’s autopilot doesn’t fail because of malfunctions such as short-circuits or bad weather conditions, or anything special happening (as opposed to humans), it fails under perfectly normal condition.
You can’t even start to compare autopilot safety performance to people performance unless you work for tesla’s marketing, for the simple reason that no human is in theory authorized to let that thing drive itself without supervision. What you’re comparing is human + autopilot vs only human.
What seems extraordinary, is that human+autopilot actually performs worse than just humans in some conditions, such as OP, simply because of the way autopilot was advertized.
This is well studied in the aviation field, and an entire subfield of psychology to study this exists called "Human Factors".
There are many many fascinating results from studies in Human factors in aviation, but one that I can point out is that humans pay less and less attention as you present them with more and more reliable automation, to a point where automation can start to decrease safety, as humans get so complacent, relaxed, and distracted (my instructor called this "flying fat dumb and happy") that when they ARE called upon to take over and act suddenly, they simply are entirely unaware of the situation and need more time to get the hang of it than there is time before decisive action is needed.
Some reading:
https://www.skybrary.aero/index.php/Cockpit_Automation_-_Adv...
https://www.ise.ncsu.edu/wp-content/uploads/2017/02/Parasura...
eg in OP. There could also be other situation where autopilot think there's a problem when there's none.
But we can say that a Tesla autopilot is never drunk, distracted, should never be speeding or driving recklessly. Crashing into an upturned truck, a fire truck, a street sweeper, a parked car, while sober, paying attention, driving carefully and attentively, is worse than human driving.
[1] https://www.statista.com/statistics/502208/tesla-quarterly-v...
[2] https://www.statista.com/statistics/183505/number-of-vehicle...
[3] https://www.driverknowledge.com/car-accident-statistics/
I think this is a case of something that seems grossly irresponsible because it's a task that humans can do extremely easily. Clearly, if LIDAR cost nothing then Tesla would happily use it in their cars. The reality is that a technology that can perform as well as a human at static object detection is cost prohibitive which is why you don't see it in any mass market vehicles. I would imagine in some cases a human driver i.e. a taxi still costs less than LIDAR.
So it becomes a question of how responsible should we make Tesla for the expectations that some of their customers may have when they release any kind of driver assistant technology? You can argue both ways that it would be irresponsible of them to release the technology that they have now without guaranteeing that every driver will fully understand its limits but also that it would be irresponsible of them not to release such technology given the number of lives it can save in its current form.
Even if it saves more than it costs it might cost more in pr and court costs than is earned from selling the feature.
It is probably impractical to sell something that saves 2 people that we cannot innumerate or name if it kills one person with a name, address, next of kin, and lawyer.