Weapons of Math Destruction
http://boingboing.net/2016/01/06/weapons-of-math-destruction...
It's easy to hide agenda behind an algorithm; especially when the details of the algorithm are not publicly visible.
http://boingboing.net/2016/01/06/weapons-of-math-destruction...
It's easy to hide agenda behind an algorithm; especially when the details of the algorithm are not publicly visible.
https://github.com/propublica/compas-analysis/blob/master/Co...
In the statistical analysis (unlike the verbiage) she is completely unable to hide the lack of bias and the accuracy of the algorithm, all of which are clearly on display in line [36]. In contrast, her verbiage somehow conveys the exact opposite impression.
Larson is still the second author so it is certainly a big question how he can present data showing no statistical correlation between race and score then have his name on an article saying the exact opposite that is clearly pushing an agenda. And as noted, one where the owners of the publication are also involved in a competing risk assessment product.
What the analysis shows is that once you know the predicted score of the algorithm, using race doesn't give you extra information. If the scores were biased then you could correct them by using racial information to undo the bias.
For more detail on that last bit, read the "What if measurements are biased?" section of my blog post: https://www.chrisstucchio.com/blog/2016/alien_intelligences_...
(The details differ a bit - I describe linear regression rather than cox models. But the basic idea is the same.)
> Black defendants are 45% more likely than white defendants to receive a higher score correcting for the seriousness of their crime, previous arrests, and future criminal behavior.
> Women are 19.4% more likely than men to get a higher score.
> Most surprisingly, people under 25 are 2.5 times as likely to get a higher score as middle aged defendants.
> The violent score overpredicts recidivism for black defendants by 77.3% compared to white defendants.
> Defendands under 25 are 7.4 times as likely to get a higher score as middle aged defendants.
> [U]nder COMPAS black defendants are 91% more likely to get a higher score and not go on to commit more crimes than white defendants after two year.
> COMPAS scores misclassify white reoffenders as low risk at 70.4% more often than black reoffenders.
> Black defendants are twice as likely to be false positives for a Higher violent score than white defendants.
> White defendants are 63% more likely to get a lower score and commit another crime than Black defendants.
Calling out one specific section that doesn't show bias doesn't magically exonerate the rest.
The algorithm is biased if it's giving the wrong score due to race or redundantly encoded race. To show that the algorithm is biased, you need to show that (score, race) pairs are more predictive than (score, ) singletons.
Line [36] and [46] both attempt to address this question. The only one of these which is statistically significant is "race_factorOther:score_factorHigh" in line [46].
The other things you bring up are interesting, but do not show bias. At best they show disparate impact which isn't remotely the same thing.
I have no interest in playing but-what-does-the-exact-dictionary-definition-say semantics games.
bias: prejudice in favor of or against one thing, person, or group compared with another, usually in a way considered to be unfair.
Then, you'll be able to click your computer mouse to solve problems.
That's certainly reasonable, but actually you're responding to a statistical argument about a statistical study, made by a statistician.
It would seem tendentious to argue that these numbers are unconnected to systemic social bias, but he is not making such an argument.
If the predictor were biased then you could build a more accurate score based on both the original scores and race_factorBlack:score_factorHigh (and other interaction terms). I.e. you'd be building a new bias in to cancel the old bias, leaving an accurate predictor.
Their analysis doesn't show that this is possible.
Orwell would have loved "disparate impact isn't bias" :)
https://en.wikipedia.org/wiki/Bias_of_an_estimator
https://en.wikipedia.org/wiki/Disparate_impact
To understand this intuitively, here's a simple thought experiment.
Consider Captain Hindsight, a predictor which returns the right answer 100% of the time. By definition, E[\hat{theta} - \theta] = 0, i.e. zero bias. (Also zero variance.)
Now suppose that blacks have a higher recidivism rate (hardly implausible, ProPublica's analysis suggests they do with p < 0.01).
Captain Hindsight - being 100% accurate and having no bias - must predict that blacks have a higher recidivism rate. Yet because Captain Hindsight predicts a higher recidivism rate for blacks, he now has disparate impact.
Seriously, you are calling standard mathematical terminology Orwellian? What's your angle here?
Your claims about the emirical means of recidivism rates do not prove what you think they prove. Different races might be misclassified at different rates for a variety of reasons - e.g., one race might be affected more by some high-variance predictor, or there could be composition effects (e.g. the pdf of blacks|high score might be different than whites|high score).
The way to factor out whether they scores are biased is to do the cox survival analysis with interaction terms. Which they did. You just don't like the result.
Could you clearly lay out the statistical argument that you believe implies that E[\hat{\theta} - \theta] > 0?
Again: I don't care about your domain-specific definition of "bias". I care about whether the end result of this secret algorithm is unequal and inaccurate treatment of different demographic groups.
This is exactly what the standard mathematical definition of bias (restricted to a given group) addresses. The authors of this article ran exactly that analysis - see lines [36] and [46].
I know that you are trying to retreat from statistics, since the stats don't support your mood affiliation, but don't retreat to "accuracy". Retreat to something vague and undefined instead. It'll work better.
As for "unequal", I don't know what you mean. Do you consider disparate impact to be "unequal"? If so, then I'm sorry to tell you that reality is imposing an unfortunate choice on you: equal or accurate, you can't have both. (According to ProPublica this algorithm chooses accurate.)
Criticizing an algorithm for revealing this unfortunate fact is like blaming telescopes for Saturn having rings.
By "equal and accurate", I mean a result that corresponds to actual recidivism rates for as many demographic groups as possible (males, females, different races, different ages, combinations of the above, etc). The analysis shows that it is substantially likely (certainly well-beyond the oft-vaunted "reasonable doubt" standard, nevermind the specifics of p-values being slightly above 0.5) that the algorithm being used doesn't provide such a result.
And no, I'm not calling standard mathematical terminology Orwellian. What I'm calling Orwellian is your describing a biased system as unbiased (by attempting to reframe the discussion around a specific statistical definition, chosen by you).