Untold History of AI: Algorithmic Bias Was Born in the 1980s
spectrum.ieee.org
spectrum.ieee.org
We expect judges to be unbiased in the (3) sense; businesses in the (2) sense; and scientific research in the (1) sense.
I have seen even professors confuse these in quite horrific ways: claiming eg., esp. that (1) is just the same as (2, 3).
This claim is common and very bad. Statistical bias has nothing to do with (2, 3). A system can be accurate (eg., an accurate record of historical crime) and still produce prejudice (eg., future over policing).
And likewise, prejudice (2) is not simply "a once-true stereotype", ie., a human-equivalent of (1) (a compu sci professor claimed this at a public lecture -- quite insane). This leads to the view that "black people were more criminal" -- no, there are no such thing as "black people".
Ie., the very concepts which prejudicial views partition reality into leads to datasets. Prejudice isnt a statistical effect, because statistics requires a measurement against a set of concepts (races, distances, etc.).
These days in the context of 'social bias' the word 'bias' seems to mean 'difference between what happens and what I think is fair'.
Statistics itself is essentially pre/post-theoretical analysis -- theory has to be added to statistics.
My use of "immoral" here is to sign-post that something is missing: an ethical theory.
I have seen esteemed people try to reduce (2) to (1) -- you may as well do so with scientific questions, ie., eliminate all of chemistry in favour of just "how often ice melts at a given temperature". We know, of course, that we do not even have data for the whole range of actually knowable states of water -- because we have theories which describe them. (And without theory, we'd not even know what data to have collected.)
Likewise, ethics is not a matter of outcomes being different whenever they are different. Ethics, like science, is a discrete constraint on theory-building -- not on data analysis.
I think this is kind of what we're talking about, though, isn't it? Does "prejudicial" mean "having preconceptions about a person based on a few observed characteristics"? Or does it mean "assuming things I think are unfair about a person"?
> My use of "immoral" here is to sign-post that something is missing: an ethical theory.
If that was the intent then it worked for me.
I am confused by this sentence. If there is no such thing as black people — and I can understand the reasoning that would lead to this conclusion, — then what is meant by the daily application of the word (black students, black Americans, black and brown community, etc.)
However, eg., if you take all people labelled "black" around the world, you'd find as different ancestries between them as-if you had taken "black and white". ie., the label "black" isn't a particular ancestry, nor even similar ones.
"Black" is something invented somewhat recently (early modern period) which was initially a hypothesis about there being human subspecies and there being such a thing as a "black people". This hypothesis is false, and it really took knowing about evolution in the 19th C., and genetics in the 20th, to see why.
Today then "black" is many things but perhaps best understood as a social caste which has been historically invented and reinforced by state institutions and latterly by culture.
It's a bit like "working class" and a bit like "tall people" and a bit like "a nationality". In the sense its blends a political grouping (class) with a physical signifier (skin color), and it is something its people identify with ("I am X").
The key issue here is that there can be no "racial dataset", in that as a scientific hypothesis races don't exist -- and as an ethical constraint, we ought never act as-if they did.
So we have these dual constraints on our theory-building and on our data collection: a scientific one, and an ethical one.
These are not constraints that can ever be deduced (induced etc.) from a merely statistical analysis of data -- and thus (2) is not (1).
(Consider, eg., that by the state creating and reinforcing a "black race" it induces commonalities between people labelled by that category. So if you actually collect data on "black vs. white" in any country, you will find "the categories you are looking for" -- precisely because youve induced these differences. It is in this sense statistics alone cannot tell you anything.)
Bad things happen if you require an AI to lie.
Anyways, this should be pretty easy to test: Just rename applicants with pseudonyms before rating.
Under that barbaric policy held as dogma the only redheads likely to survive would either avoid them entirely (and not be factored into the data) or in turn start killing in the crazed non-redheads in self defense, and being vastly outresourced looting as a matter of survival. The idiotic prophecy turned self-fuffilling as reflected by the data.
The entire premise of checking by ethnicity is also incredibly dumb - as an enforcer you shouldn't care what they look like you care if they are breaking the fucking law or not! If you lack any causology in your examinations what you have is automated superstition.
Here I dont think that helps: so long as you predict area A has more crime, area A is more policed and thus always appears to have more crime.
The issue, in my view, is not the data nor the algorithm -- no modification to either can fix the issue. The issue is the machine isn't embedded in the world, and esp. has no ability to acquire a rich understanding of the human social environment.
A fundamental aspect of understanding X is knowing what is irrelevant to X, ie., what "data" it is permissible/essential to ignore. In the case of crime, one ought ignore data from over-policing -- but this is not an effect which is present in the data itself.
There are different flavours of Machine Learning algorithms. You are referring to one of those where the learned representations are created from raw data. There are other flavours where algorithm designers manually input the important aspects of the representation. For example, a popular credit scoring agency where I live - uses the postcode as an input. The postcode in some cities is directly correlated to race and ethnicity.
The broader point is that there exists various aspects that drive bias and that goes beyond data.
One misunderstanding in this debate is that AI researchers get very defensive and retreat to, basically, "I'm not racist". What they miss was that (in most cases) they were never accused of anything like that. Everybody understands how these biases arise from the data, and are sometimes amplified by it.
What they can sometimes be accused of is not checking for the most obvious flaws in their models before publishing them. At this point, the problem is well-known and there's simply no excuse for not checking. And if that means your model cannot be released because you don't have the data, that's too bad, but your work just isn't up to purpose. Go find better data. We don't let people sell rotten steaks just because the stores are out of the fresh variety, and there's no reason to treat AI differently.
In other words, one guy who preferred Caucasian men as candidates for the medical school wrote the code with his preferences deliberately baked in. This was so obviously bad they got caught, sued, and ruled against.