I see consumer self-driving cars being introduced in three stages.
Stage 1. Limited self-parking cars. These will only be self-driving in large parking lots that have been designated as allowing self-parking. These parking lots will be at shopping malls, stadiums, concert halls, large grocery stores, home improvement stores, and the like.
When shopping at a place with a designated self-parking lot, you manually drive to the front of the store, get out, and your car drives to a parking spot at the far end of the lot (leaving the spots closer to the door for people who do not have self-parking cars). When you are done shopping, you summon your car via your smartphone, and it comes pick you up.
With a little planning, such as a little separation between where people enter/exit their self-driving cars and where manually parking people will walk, it should be possible to arrange it so that people who are skittish about self-driving cars don't come to close to them.
Stage 2. Limited street access for self-parking cars. Some parking lots and garages in cities will be designated as allowing self-parking, and some street routes leading to those lots will be designated as allowing self-driving cars to travel between designated points and the parking lots.
Essentially these designated spots will be like bus stops, except for self-driving cars. You manually drive to one, get out, and your car self-drives to a nearby parking lot. When you want your car, you go to one of these designated spots and summon it.
The participating lots, the drop-off/pickup spots, and the routes between the former and the latter would be chosen to keep the self-driving confined to a few well labeled streets, so that people who do not yet trust self-driving cars can be extra alert on those streets or avoid them.
Also there would be freeway access for self-driving cars during stage 2.
Stage 3. Expansion of stage 2. Designated routes expand, and when there are enough of them we flip to allowing all routes except designated forbidden routes. Drop-off/pickup can happen anywhere near a designated route (and later, anywhere that doesn't require using a prohibited route). Parking flips from only allowed at designated lots to allowed anywhere except designated forbidden lots.
The focus in stage 1 and stage 2 is use in limited areas at low speed, while the public gets accustomed to the presence of self-driving cars. The reason for the emphasis on parking is that for most suburban or city driving (outside of rush hour commuting) the driving to your destination is usually not overly annoying. It's dealing with parking, or carrying your purchases a long distance to your car, that is the most annoying part.
In stage 3, the network of available routes has expanded enough to span from your home to your destination, so you can let the car self-drive the whole trip.