If every car could match the best that humans can do 24x7 that should prevent almost all casualties.
I imagine the best possible driver to be a world-class rally driver who:
1. never drinks
2. never speeds
3. never drives whilst tired
4. drives without emotion/adrenaline (test pilots talk about the importance of having no adrenaline as the best way to perform)
5. Always keeps a 3-4s distance from the car in front
But I believe that when self-driving cars can match humans which is actually the hard part, then they can become 10x better because similar to airline pilots they can be trained on extreme events and so are able to respond much faster in situations that even the best unprepared human could do.