Volvo has proposed driving magnetized nails into pavements in Sweden as lane markers. Not only are these useful for self-driving cars, they're good for snowplow guidance.
Plus there's also no incentive for anyone to waylay the smart road off the physical road, because there are no evil people (i.e. with motives that are incompatible with the vehicle's survival).
TL;DR: good roads will be getting better, bad roads won't.