So the effect of a driverless car is going to be a driverless taxi. Why doesn't everyone use a taxi now? I'm sure it would cost less than cars do. But nobody really does the cost-benefit analysis of these sorts of things.
Lots of people do already use mass transit every day, which has a cost structure closer to driverless taxis (because the labor cost of a bus or train driver is amortized across more passengers).