Hacker News Reader

Comment by nyberg | Hacker News Reader

I find that calling it a `racist algorithm` doesn't really do it any good unless the behaviour was intentional. This is a case of poor training data the same as google image classification messing up with tags.

View on HN

Plenty of racism in humans isn't malicious, either, but is just a byproduct of bad training data. The outcome is bad regardless of what was intended, and it's the outcome that matters.

I think plenty of people would say it is the intention that matters far more than the outcome.

Matters how? If someone has the best of intentions but ends up creating a bad outcome, it’s still more important to fix that than to change the opinion of, say, a closeted bigot who has no effect on anyone else. (Yes, real life has more shades of gray than that, and both things are important in practice, because bad intentions don’t tend to lead to good outcomes while good intentions definitely can lead to bad outcomes.)

As an aside, “I think plenty of people would say X” is not a very good way to phrase an opinion. It’s ok to say you would say X and argue for it, rather than ascribe the opinion to some undefined group of other people.

Let's say your company decides to use AI to assist in hiring. It turns out the algorithm used is biased when it comes to candidates' race. If there is a disparate impact[1] on protected classes in hiring that's unrelated to job performance, intentions don't matter in the eyes of labor law, what matters are the effects.

[1] https://en.wikipedia.org/wiki/Disparate_impact

>This is a case of poor training data

ok we need pictures of human faces, luckily I've got all these white people here!

on edit: it was racist in result, in that it empowers a racist system, it was not racist in intention - as in the people gathering the training data probably didn't say hey how can we empower a racist system with this?

yes exactly

So were those white people exploited to develop Twitter's algorithm? Did the use of their faces raise concerns about the ethics of data harvesting?

Google exploited homeless black people to develop the Pixel 4’s facial recognition AI - https://thenextweb.com/news/google-exploited-homeless-black-...

an attempt by Google to improve its facial recognition algorithms by collecting data from people with dark skin is raising further concerns about the ethics of the data harvesting - https://www.theguardian.com/technology/2019/oct/03/google-da...

Racism as a concept has evolved in meaning. It used to only include the most severe intentional cases of bigoted behavior, whereas now it also includes less obvious biases that lead to preventable but not necessarily intentional instances of everyday prejudice and bigotry.

I am for one happy we have unneutered the word from having to reach a bar so high, it wouldn't apply to most bigotry, but it is also unfortunate for people who have not caught on and believe calling a thing racist is a damning statement of evil intent, but it really is not anymore. Or those that insist on meaning of words remaining static forever.

Reply on news.ycombinator.com