We already know that the assumption is correct for a lot of cases. Self driving cars have logged millions of miles on the road.
The only question is can we make them work in situations that are hard.
The only question is can we make them work in situations that are hard.
In other words, "it works until it doesn't" is tautologically true, but useless precisely for that reason.
On the other hand the self-driving cars are just getting started. It's entirely reasonable to suppose that they may surpass the average human in the next X years. And they won't get old and confused and eventually replaced with new cars that have to learn from scratch -- they should get more or less monotonically better. I think the optimism is warranted even if the timeline and technology is uncertain.