I also said this, self-driving will only become a thing if the roads are modified. Then I realised that this has already happened. The changes made to urban environments when cars first became a thing aren't to be underestimated. Curbs are a result of cars. Traffic lights. Pedestrian lights. Footpaths.
Back in the day, no one even considered these changes to be bad. Today if politican came up with the idea that we need to remove pedestrians (and bikes) from the streets so that self-driving cars could be safe, there would be an uproar.