The benefit to self-driving cars is not only that they are driving (hopefully) better than humans - it's also that they minimize the risk of completely erratic behavior. People don't have accidents when they're doing average. They have accidents when they're at their worst.
Last example I witnessed: At a traffic light, a car going ~30mph suddenly and without warning pulls into oncoming traffic because for whatever reason they thought "must left turn" - despite left turns being clearly disallowed.
That wasn't just a "below average driver", because that doesn't happen every day. That was somebody who was very clearly way off their game. And that's the main source of accidents, doing something clearly wrong that you likely wouldn't even think about on a good day. Or being distracted.
Taking these things out of the equation is way more important than "driving better than the best human on their best day".
And this will benefit the above-average drivers as well. (Also, the average will likely move up, so "above average human" might suddenly not mean a lot)