Tesla confirms “Autopilot” crash in Montana
ktvq.com
ktvq.com
1. Tesla's "hands on wheel" enforcement is much weaker than their competitors. BMW, Volvo, and Mercedes have similar systems, but after a few seconds of hands-off-wheel, the vehicle will start to slow. Tesla allows minutes of hands-off time; one customer reports driving 50 miles without touching the wheel. Tesla is operating in what I've called the "deadly valley" - enough automation that the driver can zone out, but not enough to stay out of trouble.
The fundamental assumption that the driver can take over in an emergency may be bogus. Google's head of automatic driving recently announced that they tested with 140 drivers and rejected semi-automatic driving as unsafe. It takes seconds, not milliseconds, for the driver to recover situational awareness and take over.
2. Tesla's sensor suite is inadequate. They have one radar, at bumper height, one camera at windshield-top height, and some sonar sensors useful only during parking. Google's latest self-driving car has five 3D LIDAR scanners, plus radars and cameras. Volvo has multiple radars, one at windshield height, plus vision. A high-mounted radar would have prevented the collision with the semitrailer, and also would have prevented the parking accident where a Tesla in auto park hit beams projecting beyond the back of a truck.
Tesla is getting depth from motion vision, which is cheap but flaky. It cannot range a uniform surface.
3. Tesla's autopilot behavior after a crash is terrible. It doesn't stop. In the semitrailer crash, the vehicle continued under power for several hundred feet after sheering off the top of the Tesla driving under the semitrailer. Only when it hit a telephone pole did the car stop.
The Pennsylvania Turnpike crash is disturbing, because it's the case the "autopilot" is supposed to handle - divided limited-access highway under good conditions. The vehicle hit a guard rail on the side of the road. That may have been an system failure. Too soon to tell.
The NTSB, the air crash investigation people, have a team investigating Tesla. They're not an enforcement agency; they do intensive technical analysis. Tesla's design decisions are about to go under the microscope used on air crashes.
Tesla's spin control is backfiring. They tried to blame the driver. They're being sued by the family of the dead driver, and being investigated by the NHTSA (the recall people), and the NTSB. In comparison, when a Google self-driving car bumped a bus at 2mph, Google admitted fault, took the blame, and Urmson gave a talk at SXSW showing the data from the sensors and discussing how the self-driving car misjudged the likely behavior of the bus.
This is what we should be trumpeting. Responsibility, honesty, and transparency. Google's self driving car project reports [1] should be the standard. They're published monthly and they detail every crash *per requirement by CA state law (thanks schiffern).
Note: These reports are currently only required in California, and only for fully autonomous vehicles. So, Tesla and other driver-assist-capable car companies do not need to publicize accident reports in the same way. Tesla may be unlikely to start filing such reports until everyone is required to do so. We should demand this of all cars that are driver-assist during this "beta" phase.
Tesla owners and investors ought to be interested in a little more transparency at this point. It's a bit concerning that Tesla/Musk did not file an 8-k on the Florida accident. We learned about it from NHTSA almost two months after it happened.
Copying a reply from earlier...
Google isn't doing this out of the goodness of their heart. They have always been required by law to send autonomous vehicle accident reports to the DMV, which are made available to the public on the DMV's website.
https://www.dmv.ca.gov/portal/dmv/detail/vr/autonomous/testi...
So the transparency itself is mandatory. Google merely chose to control the message and get free PR by publishing an abbreviated summary on their website too. Judging by this comment, it's working!
For example, here's the full report on Google's accident in February, the first one where the autonomous car was the vehicle at fault: https://www.dmv.ca.gov/portal/wcm/connect/3946fbb8-e04e-4d52...
(see also studentrob's excellent reply here: https://news.ycombinator.com/item?id=12084444)
-- "No force was detected on the steering wheel for over two minutes after autosteer was engaged," said Tesla, which added that it can detect even a very small amount of force, such as one hand resting on the wheel.
"As road conditions became increasingly uncertain, the vehicle again alerted the driver to put his hands on the wheel," said Tesla. "He did not do so and shortly thereafter the vehicle collided with a post on the edge of the roadway." --
Bruh-slow the car down then. If you're storing information that points to the fact that that the driver is not paying attention, and you're not doing anything about it, that's on you 100%.
We are in the midst of a transition period between autonomous and semi-autonomous cars, and blaming the driver for the fact that the car, in auto-steer mode, steered into obstacles, is counter-productive in the long term. You need to make auto-steer not work under these conditions. The driver will quickly figure out the limits of your system as their car decelerates with an annoying beeping sound.
That will be the communication pattern for the next two decades; we better get used to it.
You drive their cars, and you're always driving around with a third party with its own interests.
You'd think someone would have realized that a double tap could accidentally happen why trying to single tap.
The reporting on this was terrible, but there's a YouTube video illustrating the design flaw.[1]
Thing is, someone at Tesla must know this is the real culprit. But there was no public correction of their local manager who blamed the driver.
>They tried to blame the driver.
In this incident, the driver was using autopilot in a fashion that it should not be; twisting road at high speed. The driver IS at fault.
>They're being sued by the family of the dead driver
This is no proof of anything. You can find a lawyer to sue anybody over anything.
>and being investigated by the NHTSA (the recall people), and the NTSB
Well, of course they are. They're supposed to. Again, no proof of anything Tesla did wrong.
I'm not saying there isn't a problem with this. I'm saying your reasoning is wrong.
I'm also curious as to how many autopilot sessions take place every day. If there have been three crashes, such as this where the driver is at fault, out of a million then that's one thing not considered so far.
I disagree on this point. In any system that's supposed to be sophisticated enough to drive the car and but also carries a giant caveat like "except in these common driving situations..." failure is not instantly down to "user error." Such a system should gracefully refuse to take over if it's not running on an interstate or another "simple enough" situation; otherwise, as many have noted, the threshold for "is the driver paying sufficient attention" should be set much, much lower.
That the system is lulling the users into bad decisions is not automatically the fault of said user. Some blame, maybe most of the blame, has to fall on the autopilot implementation. When lives are on the line, design should err on the side of overly-restrictive, not "we trust the user will know when not to use this feature and if they are wrong it is their fault."
I don't understand why, if the car's GPS co-ordinates are known, the car even allows autopilot on anything but roads known to be within spec.
> The driver IS at fault.
I'm not sure legal liability is that clear cut.
With that said, I agree that ultimately the driver is at fault. It's his car, and he's the one that is operating it on a public road, he's the one not paying attention.
One of the laws of ship and air transport is that the captain is the ultimate authority and is responsible for the operation of the vehicle. There are many automated systems, but that does not relieve the captain of that responsibility. If an airplane flies into a mountain on autopilot, it's the captain's fault because he wasn't properly monitoring the situation.
It would be disastrous for the car to take control when it shouldn't. That itself is a giant red flag to me.
Its all or nothing when it comes to automation that involves human life.
This is the one I'm very interested to hear more about. From what I've heard, Tesla's self-driving is very aggressive in swerving to avoid obstacles -- was hitting the guard rail the result of a deliberate swerve, and if yes then was there a logic error, a sensor error, or in fact a legitimate reason to swerve?
Reasoning about what's legitimate can be a really fine line. Oncoming car, raccoon, fawn, human, doe, coyote, elk: some of these obstacles might actually be net-safer to not swerve around [perhaps depending on speed].
I'd be really curious if any of the autopilots have a value system that tries to minimize damage/loss of life. And how does it value the exterior obstacle against the passengers in the car?
It's a hell of a lot safer for Google to take fault for in a 2mph bump into a bus than it is for a case where someone died. I'd guess Tesla will ultimately settle the lawsuits against them but if they came out and admitted 100% fault, which it likely isn't, they'd have absolutely no ground to stand on and could get financially ruined at trial.
The really depressing thing is that this is first-year material for one of the most popular tertiary courses around: psychology. It is simply gobsmacking to think that Tesla thinks a human can retarget their attention so quickly. It's the kind of thing you'd give to first-year students and say "what is wrong with this situation", it's so obvious.
They don't actually believe it, but they have to make the facts fit the narrative of autonomous driving being right there.
This is actually the most distributing part to me. In these situations, graceful failure or error handling is absolutely critical. If it cannot detect an accident, what the heck business does it have trying to prevent them?
when the technology matures then bring the name back, but until then call it something else.
I am still amused that most of these self driving implementations only drive when people are least likely to need the assistance, good visibility conditions. They are the opposite of current safety features which are designed to assist in bad driving conditions. I won't trust any self driving car until it can drive in horrible conditions safely.
At most these services should be there to correct bad driving habits, not instill new ones
In any case, as somebody who has used it off and on for 6 months in all kinds of situations it is clear that it has limitations but it is also easy to know when to be worried. Its best use is in stop-and-go freeway traffic. Its worse use is on smaller curvy surface streets. It is also really nice for distance driving on freeways.
To me this seems to limit what is possible by software updates.
Does that mean that existing Tesla owners will never get 'real' autopilot capabilities by software upgrades, although I assume most expect to get them?
Maybe autonomous highway driving in good conditions will be possible with these sensors. (With the last days information, even that seems unlikely.)
Are we sure it continued under power? The Tesla is a heavy vehicle, with weight centred close to the ground. Momentum could easily have carried it several hundred feet after a high speed collision.
The system works as intended and advertised. Obviously it isn't perfect but with the fatality rate roughly equal to human drivers and the accident rate seemingly much lower it seems at least as good as a human.
Let's wait until Google's car kills someone and then compare their reaction to that of Tesla's.
"Tesla received a message from the car on July 1st indicating a crash event, but logs were never transmitted. We have no data at this point to indicate that Autopilot was engaged or not engaged. This is consistent with the nature of the damage reported in the press, which can cause the antenna to fail."[1]
When that gets trimmed down to a sound bite, it can give the impression the vehicle was not on autopilot.
The driver told the state trooper he was on autopilot.[2] He was on a long, boring section of the Pennsylvania Turnpike; that's the most likely situation to be on autopilot.
The NTSB will sort this out.
[1] http://www.philly.com/philly/blogs/real-time/Was-Tesla-invol... [2] http://www.bloomberg.com/news/articles/2016-07-11/pennsylvan...
How are these numbers compatible with each other? Was the car going at >1000 miles per hour?
Plus, I'm almost certain that Tesla's goal with the system isn't to "have an autonomous system that you have to interact with steadily", hence the 15 minute warning in the first place. They have perverse incentives - to allow it in the first place implies reliability that may or may not be there.
Naming their system "autopilot" goes beyond the marketing dept overstepping and into the territory of irresponsibility.
The "average" definition of autopilot per Wikipedia is "autopilot is a system used. . . WITHOUT CONSTANT 'HANDS-ON' control by a human operator being required."
And yet Tesla's disclaimer states:
"Tesla requires drivers to remain engaged and aware when Autosteer is enabled. Drivers must keep their hands on the steering wheel."(2)
The disconnect between the name and the actual function is needlessly confusing and probably irresponsible.
They should change the name to "driver assist" or something more accurate given that mis-interpreting the system's function can lead to death of the driver or others.
While I definitely agree on the name change, it should be accompanied by a feature change. In its current mode, and especially if people know it has been renamed, the same stuff will happen.
When you take your hands off the wheel, it turn on warning lights, break increasingly hard (depending on what it detects behind it) and sound loud warnings in the car. If it's truly an assistive feature where you can't be inattentive, the only scenario in which you stop steering is when you fell asleep.
In that case, it sounds like it might actually work more or less as it should. It's just a bit useless in its current form.
If you're not actively steering it but just waiting to take over if things go poorly, things also seem like they'd be difficult. Other than getting in an accident, how is a person supposed to know when autopilot is doing it's thing correctly, and when it isn't?
My passat has most assist systems and I like how it steers. It tries very hard to get my attention and get my hands back on the steering wheel. It barely lets me keep my hands 15 seconds of the wheel but this is perfect for what it's designed to do: keep you in the lane if you are being distracted by something (eg: baby).
Even more it has various modes to get your attention back including detecting tiredness and inability to hold the steering wheel on the lane.
If you can't drive responsibly with a baby on board lane assist isn't going to save your ass. If something happens to your baby which distracts you ask someone else in the car to take care of it, if there is nobody to help pull over at the earliest opportunity.
Lane assist will not save you from anything else but lane departure and there are a lot of other things that can go wrong while you allow yourself to be distracted because you rely on lane assist to keep you safe. That's not the intended use of this feature.
Also I find it interesting how much you assume about a person or situation from a single word.
I absolutely agree. There is absolutely no excuse for the driver to take their eyes off the road to assist a baby in the car. How would you assist a baby in the back seat while driving with a seat belt on anyway?
As I read about people using these features, I'm convinced they shouldn't exist. Yes, crazies will apply eyeliner while going sixty on a highway but I doubt driver assist is an adequate solution for such stupidity should a driver poke themselves in the eye.
You seem really defensive about a pretty inoffensive remark. Maybe you need to reflect on why you are so angry about something so trivial.
The sad thing is that Tesla has been repeatedly blaming the driver, instead of improving its technology in an openly earnest manner -- an approach that eventually made air travel the safest.
With so many sensors, car travel can eventually become safer than air travel in the future. If Tesla attitude improves, that future will arrive faster.
Here's what I did:
1 in 90 million rate: 1.1 x 10^8
Probability of No accident in one mile: 1 - 1.1 x 108 = ~9.9999999 x 10^8
Probability of No accident and driving 130 million miles: (9.99999999 x 10 ^ 8) ^ 130e6 = 0.24
Probability AT LEAST ONE ACCIDENT in 130 million miles = 1 - 0.24 = 0.76
So the true rate could really be worse than 1 in 90 million miles and they just got lucky that no incident prevented itself, which would make it more dangerous than a regular car.
And, that still includes motorcycle accident rates which shouldn't be included in a comparison to Teslas.
[1] https://www.reddit.com/r/technology/comments/4sgxv1/while_te...
On the other hand, one also needs to consider accident rate. From what I understand, based on US averages, we should have had 50+ accidents out of 100+ million miles driven on AP, but we had about three.
Moreover, I wonder how the human drivers in the statistics fare if you correct for drunk driving / using cellphone, etc. In my opinion, any autopilot worth it's name should be an order of magnitude better than the average human driver.
I'd wager that if you compare it with cars at Tesla price point -- mostly luxury cars -- Tesla would look pretty bad.
My suspicion is zero.
At that time I don't recall that Tesla made any statement about how dangerous it was to deliberately push the limit of the system to entertains viewers.
This might have contributed to make him overconfident about autopilot.
RIP
Yeah who knows. I think Musk retweeted his video. Hindsight is 20/20. Tesla can still move forward and learn from this.
People are using it in this way because they're able to, and they'll continue to do so until they're not able to. I think we agree here actually. The name isn't the problem really the issue.
* The real autopilot doesn't require hands-on-the-yoke while it is engaged, but 'autopilot' does.
* Airplanes don't have to deal with complex traffic conditions requiring interaction with other airplanes
* Airplanes do not smash into lane barriers/trees if they veer a few feet off the expected course
* "Taking control" on an airplane could safely take tens of seconds, but for a car, that is far too long.
So no, autopilot isn't good enough for cars. Cars need something much better.
Airport Runways have typically have edge lighting, and if an aircraft is off-center during takeoff or landing, can hit them and do damage[1]. Airliners land at 150mph+ so it doesn't take much deviation to hit the edge lights. Occasionally, this also happens during a gusty cross-wind landing while coupled to autopilot (auto-land), or immediately after a touchdown after a coupled landing.
[0] https://en.wikipedia.org/wiki/Traffic_collision_avoidance_sy...
[1] http://www.emirates247.com/news/emirates/etihad-plane-hits-r...
Perhaps it is true that the semi-intelligent autopilot is good enough for these trained activities, but the untrained public doesn't realize that autopilot today is not full automation.
Pilots also receive ongoing currency training. This doesn't exist for automobile drivers. Maybe some states require a written test every few years? But in the three states I've lived in, no written, oral, or in-car test it recurrency training required since the original was issued when I was 15 or whenever.
Comparing pilot certification to that of a driver's is disingenuous.
Chances of gaining control and awareness of the situation in a timely manner on an airplane are higher.
Why the hell are they so bullish on auto-pilot? I cannot see any sense in it. They must just concentrate more on the battery, charging-stations and efficiency/performance part of the equation. Instead, I am seeing more efforts are being thrown in not-so-relevant areas and that too aggressively.
Semi-automatic auto-pilot is the idea rejected even by Google. Tesla should learn fast and get back to its core business: battery powered electric cars, period and not the battery powered automatic electric cars.
I'd be happy if they prove this wrong.
edit: many typos and grammatical mistakes, sorry.
Tesla is still a tiny player in the car business and at some point a big brand will enter its market in force. They need to be bullish to maintain their image.
That said, their PR has always been overreactive. Remember all the dirt they thrown at people reviewing their car. Or how they came hard on people with their burning Tesla. Those were unnecessary "dickish". If Tesla has some actual blame to take, the tone set by their PR people is going to seriously backfire.
It's unfathomable why they don't just concentrate on their core competency (EV technology) and keep their AI experiments in R&D (where they belong).
Funnily, all of those are investing heavily in automated electric cars already, or even have products for semi-automated electric cars on the market.
Just less hype around those.
If Tesla just continues, they'll be dead in a year, and Volkswagen, which also got further encouraged by the EPA to develop electric cars, will just take over their market share.
Why? Autonomous cars =/= electric cars
With Tesla out, they can even squash the entire electric car efforts altogether and will plunder the money thus "saved" in the form of fat bonuses to the higher-ups (ala Volkswagen style), which they would have to invest in EV research otherwise.
I may seem exaggerating, but with the recent Volkswagen pollution scandal, many will agree with this. Tesla is better, Elon Musk is better in terms of innovation and their direction seems more promising for the public good. They are disrupting the fossil fuel giants who are hellbent in garnering as much money as they can at whatever costs.
I know I may (turn out to) be a foolish naive believing too much in Tesla and Musk, but I find myself helpless here.
On the other hand self driving vehicles should be a different class of vehicles that's aptly moderated by the govts. A road is a complex thing to be in as an AI backed vehicle, especially considering the lack of the communication and organisation systems used in naval and aerial vehicles.
> At Ford Motor Company, executives told me about all the pushback they received when they introduced features we now view as standard. People resisted and feared seat belts, airbags, automatic transmission, automatic steering, antilock brakes, automatic brakes, and trailer backup assist. Ford does extensive focus groups on new features that push boundaries, holding off on introducing ones that too many people don’t understand.
> Tesla doesn’t operate that way. When the company’s Autopilot software, which combines lane-keeping technology with traffic-aware cruise control, was ready last Fall, Tesla pushed it out to its customers overnight in a software update. Suddenly, Tesla cars could steer and drive themselves. The feature lives in a legal gray area.
[1] http://fortune.com/2016/07/11/elon-musk-tesla-self-driving-c...
Exactly!
I'm not sure of the legalities in the US, but in the UK I'd suggest that there are a few things that they'd fall foul of:
o advertising standards, its not an autopilot o trading standards, not making a safe product/warnings [0] o death by dangerous driving [1] o driving without due care and attention [2]
[0] There are layers of consumer protection that enforces things like no sharp edges, doesn't give off nasty vapours, does what the marketing says it does.
[1][2] these are crimes for the driver. If you are found to have been driving in a reckless way, non concentrating, or not having hands on the wheel, that your fault as a driver. very large fine and time in jail.
especially as the tesla has lots of sensors to prove what the driver was doing.
This argument is completely bogus. No driver in the nearby lanes is aware the other driver is drunk, a bad driver or emotionally unstable.
The entire idea of beta testing is to have your system on an "uncontrolled environment". The system has bugs, as any beta system does, and whoever uses it should be fully responsible for it. Yes, fully responsible.
Tell me what's the difference between using a beta system and not caring (leaving hands off the wheel, etc) and drunk driving? On both situations the driver is choosing to put everyone at risk, nobody knows about the fact until after an incident and if there is an incident, who's fault is that? The beer company? The car company?
If the driver assistance made cars crash because it ignored a command from the driver or something like that, then sure... it's 100% Tesla's fault. But clearly on all of these accidents the drivers were careless to the extreme by leaving hands off the wheel for several minutes.
We can discuss if they should try to improve how they handle people that remove their hands from the wheel, but that doesn't change the fact that ultimately the choice to be careless was made by the driver, not the car manufacturer.
> We can discuss if they should try to improve how they handle people that remove their hands from the wheel, but that doesn't change the fact that ultimately the choice to be careless was made by the driver, not the car manufacturer.
Yes, the driver is to blame for that, just as tesla is to blame for allowing that to happen in the first place. They have sensors to detect whether the driver has his hands on the wheel, and yet they allow the vehicle to keep going for minutes in a situation it can't control.
There's plenty of blame to go around.
The latter is a clearly defined criminal offence whereas the former is currently unclear. While the driver is certainly liable for reckless driving, that the marketing seems to have suggested that the feature is usable, is a fact.
Signing an acknowledgement means there's something to acknowledge, like "we're not totally done with this." Any driver on the road can reasonably expect that any nearby car was sold fully functional.
The truck that the one Tesla crashed into was not (I believe) injured, although the truck's owner experienced property damage caused by Tesla (my opinion). The truck driver could have been injured or killed. If it hadn't been a truck, it could have been a white mini-van, and there almost certainly would have been injuries or deaths.
This think belongs on a test track, not crawling up my bumper.
Similarly, if you provide and advertise an Autopilot feature that isn't autopilot, it is your responsibility to ensure the driver is attentive, has their hands on the wheel and that you pushed a not-broken implementation to them. Other manufacturers have done this, yet Tesla hasn't.
If the autopilot's confidence was dropping that rapidly, why did it not slow the vehicle? If it hits zero confidence it should halt completely until the driver restarts the car in manual mode, and possibly disable itself until checked out.
So it's not actually stopping but giving you a clear signal that you should do something and would wake you up if you fell asleep. I don't know if it will come to a stop if you also ignore this warning, because it's not something you want to test with other participants on the road behind you.
I think that's the important difference: This system actively disencourages the driver from fully relying on the system while Teslas implementation does not seem to do it.
The clue stick comes in many forms.
Tesla is the only car manufacturer that does not have a gradual emergency braking system implemented.
Hey, while we're at it, why not make the car flash its hazards and call an ambulance for its non-responsive driver?
Given that the sensors can't detect a semi completely blocking the road, I'm reluctant to make any statements about what the Tesla can and can't detect.
1: https://www.amazon.com/Our-Robots-Ourselves-Robotics-Autonom...
We can expect none of these things from the average driver - they're more likely to be watching Harry Potter on their iPad
Ok, it is still a risk, and a bad accident may ensue. But most of the times there is a vehicle stopped on the road the traffic behind it simply stops as well, and in most places there is a hard shoulder that you can park on.
It's not as if cars don't strand for other reasons (engine failure, driver incapacitation).
If gentle deceleration "causes" an accident then the problem is with the attitude and behaviour of the following vehicle, not with the deceleration.
Pretend we're in the car together. We're traveling at 70mph on a two lane road at night. I cover your eyes with my hands on a curvy road. Assure me we will stop in time without hitting something.
Traditional real-world enigneers have been drilled in this for longer than living memory. When the software guys decide to bring their software appraoches to the real world, this is what happens.
From 12 days ago: https://news.ycombinator.com/item?id=12013713
2014 Model S got 5-star/82%[0] (where 100% is best) adult occupant protection on EuroNCAP. For comparison, the smaller, lighter, cheaper 2015 Honda Jazz got 5-star/93%[1]
Other cars are significantly better than this [2]. E.g. 2016 Alfa Romeo Giulia 5-star/98%, 2016 Toyota Prius 5-star/92%, 2015 Volvo XC90 5-star/97%, 2014 Porsche Macan 5-star/88%. I could go on, but the Model S is not the safest car ever tested - it has approximately the same level of crashworthiess as the 2014 Smart Fortwo (4-star/82%) [3] according to EuroNCAP when looking at adult occupant safety.
Cars have come a long way in the past decade or two though - its amazing to see "small" cars like the Jazz where they dont even get a cracked windscreen when basically driving into a wall at 40mph. For comparison check out the 2000 Citroen Saxo fold up like a tin can... http://www.euroncap.com/en/ratings-rewards/latest-safety-rat...
[0] - http://www.euroncap.com/en/results/tesla/model-s/7897
[1] - http://www.euroncap.com/en/results/honda/jazz/21484
[2] - http://www.euroncap.com/
These ratings also don't test the assist features in-depth.
Don't get me wrong - Tesla engineers are talented engineers. The culture of the company however is not suited to building mass-production cars. It's trying to put the Silicon Valley philosophy of rushing towards the future ahead of taking measured, reliably proven steps.
You don't build buildings or car structures via an agile methodology using customer feedback. Waterfall-style approaches are used because hammering out the long tail of risk is important. Car software technology must necessarily be no different.
From the article:
"We specifically advise against its use at high speeds on undivided roads," it said. Tesla states clearly in its owner's manual that drivers should stay alert and keep their hands on the wheel to avoid accidents when the Autopilot feature is engaged.
Tesla could easily have the "autopilot" turn off if no hands are detected on the wheel for, say, 10 seconds. Putting a blurb in the instruction manual about keeping your hands on the wheel but then allowing users to use it hands-free is broadcasting, "Hey, you don't really have to do this"
Yes. I hear you. If I, as a non-Tesla driving, regular motorist got grievously injured by a Tesla running autosteer, I bet I'd be able to choose from the nation's finest injury lawyers.
There's an old saying: your freedom to Tesla autosteer your vehicle ends at the tip of my bumper. (Or something like that.)
You don't have to advise; you could mandate. The car knows how fast it's going and where it is, and with a little help from its Internet connection, can know when it's on an undivided road.
What is then the point of having an autopilot?
Similar with the garage door incident - "Well, no-one said it should be used without full supervision! It needs to be watched!" ... like in the Tesla videos where the owner summons the vehicle as he's walking out the front door, or hits auto-park... and walks away.
IIRC think the Model X had a problem with the headrest already that resulted in full recall.
I hope he's not putting too much hubrys in the AutoPilot thing and try to enforce it at any cost. Soon there'll be so many corner cases, AP will only "best used while parked with nobody inside nor outside the car, preferably with engine stopped too".
The reason I liked Tesla/Musk before is that he pushed quality and safety Unconditionally. I'm starting to think it was mostly nerd honeymoon phase. Either revamp the logic (sensors-recall and software) or disable it.
This is the real danger of autopilot: not that it is unreliable but that humans are lazy and don't think rare events (like dangerous situations that result in a crash without intervention) don't happen to them.
I'm not sure autopilot is a good middle ground between assistance and full automation exactly because of what these accidents show. It basically makes the typical ride so uneventful and mind-numbingly boring drivers stop paying attention and become unable to react appropriately when a dangerous situation does occur.
Cruise control had similar problems: it reduces speeding-related accidents (because drivers are less likely to micro-optimize their speed) but it increases the risk of rear-ending because drivers don't have to stay as alert for the majority of the driving (and then miss those situations in which they need to react quickly).
Humans are not good at staying attentive for extended periods of timing, especially when nothing ever happens. Self-driving cars can fix this by removing the need for human intervention, but autopilot seems like one of those technologies that should be safer in theory but ultimately fails to consider the human factor.
Why doesn't the "autopilot" turn off if no hands are detected on the wheel?
Until full driver attention is no longer needed, enabling "autopilot" and saying in the manual "PS. you still need to stay as alert as you normally would and keep your hands on the wheel" (i.e. no benefits from having this feature) is just asking for accidents.
Imagine, as you suggest, it does turn off when there's no hand on the wheel. People are just going to rest a hand on it and still not pay attention. What's the next step, tracking people's eyes? Or shall we just not enable autopilot in the first place, if it can't do its only job?
That 30 minute trip could end in disaster or, using something like "autopilot", it could end with me pulling into my garage without issue. I tend to think that it will be very useful in these types of situations.
I understand what you're saying, and that it is unlikely that something will happen, but Tesla still recommends full attention in this scenario. You need to be fully attentive at all times when driving, even with an assisted driving system. Tesla states this clearly in their manual. If something were to come across the road that the car were not prepared for, it may beep at you, or it may not.
This seems to be what happened to the Florida driver. He likely did not think he was abusing the system. He probably crossed that road hundreds of times before on autopilot as part of a commute. And, neither the driver nor the system pushed the brake.
If I were you and got into an accident, I know afterwards my brain would search for a reason why this happened and how I can help it not happen again both to me and to others. The solution is to not drive when tired, and to use the system as designed. An additional thing we can do is put in a few measures to check that the driver is still alert, like requiring hands on the wheel at more frequent intervals.
Sane people don't set out to drive while medically incapacitated. That doesn't mean it can't happen while on the road (heart attack, TIA, severe allergic reaction to e.g. a bug).
The current SV engineering climate -- youth beats experience, move fast and break things, release early and often -- works great for photo sharing apps but not great when lives are at stake.
SpaceX hasn't had people in their vehicles yet. Maybe they're be more careful when they do. Or maybe not. I can't find the quote right now but I think it was Sagan who said that part of what makes exploration so captivating is that it is dangerous, maybe we shouldn't make safety the number one priority because it might slacken our awe.
> The driver in Montana was headed from Seattle to Yellowstone National Park when he crashed on a two-lane highway near Cardwell. ..."It's a winding road going through a canyon, with no shoulder"
From the Tesla Owners Manual:
> Traffic-Aware Cruise Control is primarily intended for driving on dry, straight roads, such as highways and freeways.
...
> Autosteer is particularly unlikely to operate as intended in the following situations:
• When driving on hills.
• The road has sharp curves or is excessively rough.
Humans are the guinea pig for Tesla. They want this data of their system driving outside its normal bounds. Otherwise yes Tesla would disable autopilot on unapproved roads.
* Real life people who aren't engineers don't read the manual and they don't read notification pop-ups.
[0] https://www.google.com/maps/@45.8317611,-111.9106714,3a,75y,...
Wow, really?
This is not the Autosteer's fault, it's pure negligence
https://www.google.com/maps/@45.8317611,-111.9106714,3a,75y,...
People with no respect for the danger of driving are going to get hurt, with or without driver assist technology.
Most people don't read the manual unless what they're doing actively doesn't work. Especially with press surrounding Autopilot saying it's basically the best self driving car and is completely trustworthy (even if they don't explicitly state that, it's still implied), it's not unthinkable that someone would completely trust the car and not worry about sensors. The car should be much more proactive at getting the user's attention, either by actively slowing the car down to some safe speed with cautions on and/or significantly lowering the time limit of taking their hand off the wheel before any indicators appear. I'm not sure about the former, but the latter is done in every other car that has this kind of technology (e.g. cars with Lane Assist) and the time is very low, around 10 seconds.
It's clear from their PR response with the semi-truck accident that they see fatalities as an acceptable consequence as long as they can say they have fewer accidents per mile than 'un-assisted' drivers.
The question is: do we share their belief that this is ethical?
The other (fatal) crash involved an inattentive driver watching a Harry Potter movie:
https://www.theguardian.com/technology/2016/jul/01/tesla-dri...
The claim claim from the truck driver hoping to avoid responsibility for crossing a highway in an unsafe manner.
I think you are wrong that this the first incident where autopilot was involved with an inattentive driver. In the fatal crash responders found a portable DVD player inside the car and seemed confident it was in use at the time of the crash.
A traditional car won't get more than a few hundred meters if the driver on the freeway has a stroke, say. A car on autopilot could travel hundreds of kilometers and end up in a schoolyard at freeway speeds.
What is the chance that a Tesla car fails to continue safely and eventually pull over after the driver carks it? 0.1%? 1%? I doubt anywhere near 99% of strokes while driving end with the car just slowing down for a hundred metres and stopping safely in the middle of the road. I'd be surprised if the majority do. Not that it's a common occurance, but if I picture a typical stroke while driving alone, I'm picturing a car veering slowly off the road and hitting something at maybe 50-75% of the speed it was originally travelling.
Tesla's self-driving would have to be abysmally bad to not beat that, and given we're only discussing 3 accidents so far, I don't believe that to be the case.
Just the stats, ma'am. Are they safer or not? If so, then it's fine by me. What we gain from self-driving cars in terms of real world usage data is extremely valuable.
actually try to let both computers and people excel where they do best, and massively decrease road accidents?
For example, it does not handle any crossing traffic, nor stationary obstacles as well as debris on the road. Off the freeway it is entirely the responsibility of the driver to keep the car safe, the autopilot cannot (by design) handle most of the situations.
The discussion of having the hands on the wheel is a bit a red herring. Having the hands on the wheel limits the amount of distracting things you can do while driving, but the key point is paying attention to what is going on. Hands off on an empty highway, why not? But when a truck appears on the horizon with some potential to cross - better be prepared to take action. The same applies to small country roads, and especially when the autopilot starts to warn the driver.
in SUV (Model X)... are people trying to win Darwin award?
One of the main selling points is that commuters can save time (the focus on self driving cars, not trucks/rail). However, long commutes are mostly a constraint imposed by suburban sprawl and associated planning failures. The same applies to automated delivery mechanisms from shops, again of limited use in sufficiently dense communities. In such a scenario, this feels a lot like optimizing 1% of execution time in a program.
For large scale transport of goods or people, the same situation applies for the economics - the labor of the driver should be a small fraction of the operating costs. There could be some safety benefits, but this remains uncertain. I found https://www.bloomberg.com/view/articles/2015-05-28/cars-will... quite interesting in this respect.
Seeing what happened with the invention of the automobile, I won't be surprised if self driving cars become the norm somehow. In the most extreme case there might be an outright ban on non autonomous cars at some point in the future. This is something I certainly don't look forward to.
Having the hands on the wheel only gives the car manufacturers leverage in denying there is any problem on their end.
I thought we were prepared to deal with the inevitable crashes, but based on the chicken littles here, maybe I was wrong.
Good question. It's not safer. Tesla's quoted statistic of one death per 94 million miles driven in the US is not as useful as it could be. That includes motorcycle deaths and roads and conditions in which the Tesla AP is not likely to be engaged. When you limit it to divided roads, the rate is more like 1 death per 150 million miles driven [1], which makes the Tesla less safe than your average American vehicle, and that still includes motorcycles.
[1] https://www.reddit.com/r/technology/comments/4sgxv1/while_te...
I think it is. Given the number of Autopilot miles driven, you'd only expect one death so far, so that we have one doesn't really tell us anything.
However, injuries from car crashes are (fortunately!) much more common - roughly 60 times as likely. So you'd expect to have about 60 Autopilot incidents resulting in injury by now. I don't really keep up with Tesla news but, based on the reaction to this story (where the guy wasn't even injured), I'm going to assume the actual number of injuries is tiny.
What do you think?
I think it's not safer based on existing data. Musk was pretty quick to tout that autopilot is safer over 100 million miles driven. It turns out his statistic was not as useful as it could have been. There was no mention of conditions in which autopilot is not likely to be engaged such as adverse weather or non-divided roads. Plus some deaths are from motorcycles, which, it seems, should somehow be excluded from the comparison.
On top of that, it might make more sense to compare the Tesla models' safety records to cars in a similar pricing class. Comparing a Tesla to a beater from 1990 isn't so useful. So there are different degrees of comparison. Tesla's headline-ready numbers do not tell the full story.
I don't feel badly about drawing different conclusions from the same data Tesla used in its response to the incident. I think they overlooked a few things. You're certainly free to intuit what you like.
> I don't really keep up with Tesla news but, based on the reaction to this story (where the guy wasn't even injured), I'm going to assume the actual number of injuries is tiny.
Tesla does not make this information available, so I don't think it is worth speculating. Effort would be better spent encouraging regulators to demand more detailed reports from companies that sell vehicles equipped with driver-assistance systems. Until all companies are required to report such information, Tesla is unlikely to do so. We can turn the screws on Tesla too, and I think they would be served by lobbying for tighter regulations to create a level and safe playing field for car companies and drivers alike, but I wouldn't count on them to spend much extra effort on holding themselves accountable in such a fashion. They're already having a tough time meeting their production demands, and Musk is still setting aggressive targets for complete autonomy that no other car company, including Google, is even close to. I think Musk has said full autonomy is 2 years out, whereas Google says it's between 3 and 30 years. Mobile Eye has a 5-10 year plan.
[0] http://www.thewrap.com/jeffrey-katzenberg-in-life-threatenin...
I also seriously doubt it's much safer than driving a normal car in the same situation. Musk threw out a bullshit comparison with general vehicle fatality stats. But Autopilot is used in a situation that's safer than average travel--highway driving in good weather. Most deaths happen at intersections--not highway.
I had a professor in college who specialized in airplane cockpit design. Her philosophy was that everything is a design failure. If the pilot pushes the wrong button, the design must be fixed to keep that from happening. It would seem to me that principle applies with even more force when we're talking about consumer products operated by random people rather than trained pilots.
Paving the way for autonomous driving by making crashes something that people expect sometimes?
Or putting people off autonomous driving by making people think crashes are not acceptable at all, ever?
I believe their thought is that the closer we get to complete automation, the less likely a driver is to remain aware. We're only at roughly 10% automation right now and drivers are already taking their eyes off the road. When we get to 90%, humans will have an even tougher time retaining attentiveness. Watch their talks to be sure.
Google's self driving car group gives some awesome transparency reports every month, including details about every accident [1]. It's like they're ready to become their own company.
Fortune had a good article critiquing Tesla's strategy vs. Google's and other car companies' [2]. The author says Tesla is being defensive and resistant to public critique. Other companies expect pushback from the public and incorporate that into their product offerings.
[1] https://www.google.com/selfdrivingcar/reports/
[2] http://fortune.com/2016/07/11/elon-musk-tesla-self-driving-c...
Google isn't doing this out of the goodness of their heart. They have always been required by law to send autonomous vehicle accident reports to the DMV, which are made available to the public on the DMV's website.
https://www.dmv.ca.gov/portal/dmv/detail/vr/autonomous/testi...
So the transparency itself is mandatory. Google merely chose to control the message and get free PR by publishing an abbreviated summary on their website too. Judging by this comment, it's working!
For example, here's the full report on Google's accident in February, the first one where the autonomous car was the vehicle at fault: https://www.dmv.ca.gov/portal/wcm/connect/3946fbb8-e04e-4d52...
Hahaha I guess so! :-D
There should be a similar requirement for transparency of driver-assist vehicles. Clearly, the companies are not giving this up themselves until they're all forced to do it at once.
>The crashes are calling the safety of such automatic driving features into question, just as they're being incorporated into more and more cars on the road.
Tesla's failure in this court harms the entire concept, not just their own brand. The average person isn't going to distinguish between Tesla's inappropriately named 'autopilot', which is not 100% automatic, and Google's, which is.
There is plenty of time for Tesla to reconsider its strategy and for Google to continue building on its good reputation.
Doesn't sound like the recommended use case for autopilot? I thought you were only supposed to use it on a limited-access throughway?
Anyway, good to hear no one died this time, but both of these accidents do sound like there's a lot of user culpability involved. It seems like it would be reasonable on Tesla's end, since all Teslas know where they are, to limit autopilot to only roadways where it is reasonable to use it.
Does the autopilot routinely alert for short duration events where it's confused but the situation passes quickly and it's not a problem or is there a very low false positive rate and an alert is pretty much a guarantee that you'll need to take over?
Basically, does the system train you to ignore most alerts because of a high false positive ratio?
Thank goodness no one was hurt.
But I have to ask, what was the driver thinking? Autopilot late at night on a winding mountain road with no shoulder?
A question for Tesla: Why isn't Autopilot disabled when it detects dangerous conditions? TFA said it alerted the driver to put hands on wheel. If there is no response, what is the protocol? At what point will the vehicle attempt to pull over and stop (for instance, in the event of a medical emergency or conditions are simply too dangerous)?
Or assume they wanted to do the right thing from the start, and they don't have this dataset. It's not as if we don't have OpenStreetMap (or as if they couldn't buy a commercial dataset) in which you can easily filter out straight highways.
With this it could probably even work without being attentive as a driver, which would be a killer feature over the current state of affairs (currently it has no benefits whatsoever, if you follow the instructions, which is why probably almost nobody does).
I think they want the data of drivers attempting to drive with autopilot on these roads.
Tesla does, I've read, restrict how fast autopilot can drive on unapproved roads, so it appears they already have them mapped out.
They're using customers as guinea pigs for their tech.
I imagine Tesla is generating more sales with a usable autopilot. It generated quite a bit of media hype and they are having trouble keeping up with production demands.
So, Tesla chose to introduce this feature to increase sales and get more data on how drivers interact with the system. It seems excessively risky to me. I'm not building a self-driving system myself, nor running a huge company, so I can't say much about what it takes to do that. I can say I would not like to drive on the road next to a vehicle that can unexpectedly make the wheel turn, forcing its driver to be corrective at a moment's notice.
Thus you have people watching movies, falling asleep, or driving on dangerous roads using a technology that nowhere near delivers what the name promises. The result is people are dying and getting hurt by an immature technology being pushed too hard by an irresponsible company.
An autopilot system is defined as a system that assists, but does not replace, the human operator of a vehicle. Therefore Tesla's system is, in fact, an autopilot.
That's the clear opposite of what Tesla's manual suggest. If it required hands-on wheel and constant attention, call it what it is -- a driver assistance system, not autopilot.
The downside of this is that when you get bad publicity it's going to be much bigger as well.
That's the price of celebrity.
I wish Tesla was more open about the way the actually test their Autopilot software.
Of course they were aware of that situation. They chose to deprioritize that to reduce the incidence of false positives that came from overhead highway signs. Nothing wrong with attempting to squash false positives, which can lead to deadly situations themselves. But there was nothing preventing Tesla from doing adequate testing to determine what unintended consequences their software modifications would entail. They don't have to wait for someone to get decapitated to realize that there's a trade off in reducing false positives from overhead signs.
Airbags occasionally kill or injure people also despite on balance saving lives. So there is precedent for this sort of math, but this is a very different kind of technology and not an ordinary engineering tradeoff.
I've heard commenters opine that this "could mean the end of Takata as a company." But I haven't heard anyone say this "could mean the end of airbags as a feature in cars."
I guess what the Tesla autopilot needs is a lot of good publicity every time it avoids an accident which a human driver would otherwise have actually caused.
But where would anyone get the counterfactual for those cases? Does anyone even have information about that?
Sounds like a plan! Let's start right now. Here's an example of the autopilot dodging a reckless driver and avoiding a collision that many human drivers wouldn't have avoided: https://youtu.be/9I5rraWJq6E
Even if that happened, people will cling to the negative news much more than the positive news, aka Negativity Bias. Even if Tesla has a stellar record overall, one bad incident, especially with a somewhat unknown technology, will be much more in people's minds than the countless positive incidents.
And yet, people are killed virtually every day by car accidents, but nowadays there's no serious legal effort to ban all cars.
How did humanity overcome its fear of cars in the first place? That same process is the one autosteer needs to undergo.
https://www.youtube.com/watch?v=9X-5fKzmy38
and this
To be honest I don't feel comfortable with that kind of data collection anyway. I'll never even think about getting a Tesla until they give us an option to turn their tracking systems off. It's creepy and unnecessary.
Heck, mine the data from streetview and disable activation on roads where it can't work 100%
Maybe I'm drinking the Fortune coolaid. Their article seemed logical to me [1]
[1] http://fortune.com/2016/06/22/tesla-elon-musk-solarcity/
My wife calls me crazy for not even using cruise control on our car.
Meanwhile, many humans who are NOT using cruise control do the opposite: they don't push their foot enough to counteract gravity and force a downshift, and just accept that they lose speed going up a hill.
If the traffic is dense enough, what can happen is that all of sudden your cruise control is downshifting and jumping forward just as you are overtaking a car that is itself losing speed. Potential for a rear-end collision at highway speeed.
Of course this isn't an issue if cruise control is only used in very low densities of traffic. But unfortunately it seems to be used wherever people feel like their feet need a break.
So keep your foot "active" and ready for use, even with cruise control.
Personally I lightly rest it on the gas. (Not hard enough to actually do anything, but in the right position to react to things.)
The problem I see is, Tesla is the only one actively testing autonomous driving.
The fact that over 100 million miles driven, and we're finally starting to see accidents, is a good sign.
And that is the problem right there. Tesla is "testing" a feature that is being shipped to consumers in a finished product. They are providing the most minimal of safety training regarding the feature, and not even enforcing simple technological measures to make the car safer in this mode (e.g. making sure your hand is on the wheel. They have the sensors in place that test this already, and yet they do nothing with that data).
So essentially, the users are beta testers. They are testing the limits of the sensors. Looks like they don't work on white colored trucks blocking the highway. Sorry you were decapitated, we'll fix that ASAP though!
This is what happens when you bring the "work fast and break things" mentality to an area where "break things" means "people die".
To be clear, I am very pro autonomous car technology. I am a robotics engineer. I've worked on these systems. But this is the exactly wrong way to bring them to market. As more people die due to tesla's autopilot not being a true autopilot, it will color the public's view to reject this technology before it's even mature enough to save lives.
No, they aren't.
They may be the only ones using consumer-operated vehicles as their test platform for the particular degree of automation that they are testing, but there may be very good, safety-related reasons why other people aren't doing that.
But, I understand that you mean behind the wheel. Which is fun because the Google cars do not have a wheel. As I'm sure you may know, instead of a traditional steering wheel they have horizontally mounted, valve wheel with a handle. It looks a little bizarre when you first see it, but I would imagine it could be a safer, smoother alternative to the current standards steering wheel.
Unfortunately, I couldn't quickly provide an image; I've seen it multiple times on my walk to and from work.
There is a huge difference between Google operating a handful of cars in certain areas and Tesla selling 10-20 thousand vehicles per month that can be operated by untrained consumers anywhere in the US.
Tesla is using its customers as data-generating guinea pigs. In return, Tesla may become a guinea pig itself by showing other companies how not to progress towards autonomous vehicles.
No it's not because the human safety record is approximately that. Which means this Autopilot did nothing, and the humans did everything.