Driver wasn’t paying attention, not surprising. You have to actively ignore the warnings to pay attention or hack together a rig to put torque on the wheel.
Let's say you're paying attention with all your might. You see something up ahead, like an overturned tractor trailer. You expect autopilot will sense it, but you're preparing for it not to. But how do you know it's not going to sense it? Well, obviously, when it hasn't started reacting in a reasonable time. But then it's too late!
In order for a human to have time to figure out that the software isn't working it would have to routinely operate with (much) slower reflexes than a human. But then it would be useless, as you can't slow down normal driving, and in any case, that's not how they designed it.
The hypothesis that people have unreasonable expectations of something called "Autopilot" seems unnecessary and irrelevant to me.
It's even more egregious with Tesla, who is already shipping a half-baked self-driving feature in its current state where accidents like in the article are inevitable.
I could even imagine the windshield outlining everything that the Autopilot sees. If you don't see an outline around a object, you know the Autopilot isn't aware of it. No idea how technically feasible that is.
People say "make a program that does <thing> for me". And maybe you whip up something that does 50%, or 80% or 99.5%. But that only makes them more unhappy when it fails. A partial solution is no solution.
So, you have to come up with a model for humans to be augmented by the software. Rather than trusting it to make the decisions, the software needs to take a large amount of data and clarify and distill it so humans can more easily make the decisions.
But people always want to avoid making decisions. That's why managers/leaders have so much power even though they tend to be despised and seem incompetent.
And if you saw it on its side, why wouldn't you at the very least change lanes?
It's fun to bash on Tesla whenever autopilot fails but they do make it very very obvious and those of you saying otherwise either don't own a Tesla or really don't pay enough attention.
And if so, how does it make sense to have a feature that's always wrong to use?
I am however, doubtful think it's really reasonable for humans to have this expectation of 100% attentiveness given an autopilot that 99.9% of the time doesn't crash into things .
The simple answer is that you take over when something absurd is happening in front of you, like a trailer flipped on its side blocking your lane.
You don’t wait till the last moment to take over. You would immediately move over to the right lane, manually. AutoPilot helps in this case by keeping you perfectly in lane while you check your blind spot before getting over. That’s how it’s supposed to work anyway, and that’s how I use it.
Turns out that driving is 99.9% utter monotony and AP actually drives better than me during all that time.
you're conflating not paying attention with not having torque on the wheel