Pedestrians and motorcyclists are supposed to play chicken, one supposes. The self-driving problem requires general intelligence and a theory of mind, both unsolved problems.
Self driving only has to be better than people drivers, statistically, to be a benefit to society. This probably isn't a very high bar.
Further, I think a variety of factors, including politics and liability structures, mean self-driving cars will have to be radically better than the average human driver. For example, each car-caused death is seen as the responsibility of some human. But every Waymo death will be seen as the responsibility of Waymo. Responsibility for the current 100 deaths per day is diffuse, and so accepted. But if it's 40 deaths per day for Waymo and 60 for Musk, nobody's going to shrug at that.