Thats difficult. People who talk about "self driving cars 5 years away" aren't usually talking about mythical level 5 self driving that is actually 40 years away. They are talking about level 3, which basically means, "end to end travel, in 90% of situations".
Just like how 1/10 times you might not be able to flag down a taxi cab, today, 1/10 times you won't be able to get the self driving taxi to go EXACTLY where you need to go.
But it will work for most situations, most of time, and thats good enough. (IE, imagine if 90% of taxi drivers or truck drivers were made obsolete. That is industry destroying)