Yes. but in my limited experience with ML, "the data is good" usually isn't the

>> most likely

explanation :-)

> This is what happens with loan or crime data sets, and it made plenty of noise in the news recently - but only the kind of noise in which people say it's obviously the algorithm that's broken, because it doesn't fit the "polite fiction" they'd like to believe.

People say the algorithm is broken because it's illegal to discriminate on the basis of race, and these algorithms were sneaking "discriminate by race" "in through the back door".

Calling the law a polite fiction won't get you very far when the judge issues an injunction against using your product because it's racially biased.

And the judge doesn't care how machine learning researchers define bias. He cares how the law defines bias.

So, if you're diddling around on your computer in your own time, then I guess the algorithm isn't broken. But if you're building a product you want to sell to courts or insurance companies, the algorithm very much is broken.

That is, unless you think "the law is a polite fiction and bias means only what ML researchers say it means" would be a winning argument to lift an injunction against your product. If you're going to make those arguments in front of a real judge, let me know; I want to see the bulge in that judge's forehead :-)

No. People say the algorithm is broken because it obviously is broken.