Couple this with the fact that most people are still concentrating on the road while autopilot is in action. So in effect you still have a human driver, they are just not fully engaged with the cars controls.

I know when I tried my friend's auto-pilot that I was concentrating 10x more on the road than I would be normally. Obviously if I got more comfortable with the tech then I would be concentrating less but I can't imagine that I would ever stop paying attention unless I knew the system was fool-proof.