What I’ve seen is differential comparisons, eg, comparing the rate of white detection to black detection or the difference in certainty scores on each — but I’d really appreciate it if people could show me the actual certainty numbers on black faces so I can see if it’s failing to recognize, misrecognizing, or just less sure then white faces.
Regardless of whether the higher error rate is a combination of race, gender, or both, it's still a huge issue. Granted, that study was from a year ago, and other companies have since improved their facial recognition systems. But an overall precision/accuracy/f1 score doesn't mean much when accuracy varies that much by group. Sure, you can market it as "accurate on white males", but you can't market it as "accurate"