While I agree with much of what you say, I think we can have autonomous vehicles without having to be 100% independent of human intervention.
It will require an effective "early warning system" which foresees issues it may not be able to process --however, this will require more inter-automobile communication and extravehicular communication (GPS, other inputs). Relying on only what the vehicle has on-board will probably not be good enough, for now.
These supporting systems would communicate "trouble ahead" (ambiguous information, incomplete information, lack of information, unknown information, etc.)