Hacker News Reader

Comment by Y_Y | Hacker News Reader

I've dealt with a lot of badly labelled datasets, but I can't say I ever saw one maliciously labelled by racists.

There are other more subtle problems that may contribute. As an amateur photographer I've been told that most cameras are calibrated for light skin. If you aren't careful with your setup you'll get bad definition in shadows and relatively very dark areas. If you have badly calibrated/exposed photos then very dark faces, whether of sub-Saharan people or gorillas, may be poorly distinguished. I bet there are loads of unfair, hard-to-catch issues like that.

By the way I don't mean to say that you're wrong, but just to offer a Hanlon's Razor-type counterpoint as it occurred to me.

View on HN

This is usually a good example of societal bias compounding.

It's possible that nobody directly involved in this was actively racist, but we're building on centuries of racism in every facet. So a little bit of bias in each technology and data set involved, not to mention the engineers and their testing, just adds up

>but we're building on centuries of racism in every facet.

Nooo. We're building on techniques that have worked well enough up til now, but people don't bother to dig in a d come to terms with what's actually going on.

Not everything is racist. It's more a tyranny of 'good enough' and convenience.

So you're saying there's no bias in the data sets , or in society? Do you think that someone who thinks there's no bias in their dataset would then dig in to see what's going on? Do you think people are going to recognize the negative biases in the data if they don't face those issues themselves?

Bigotry is part of society, because it's existed for so long and still does. It doesn't mean that everyone is actively racist. It just means you can't just say there's no bigotry affecting things, and in saying there isn't, you're now blinding yourself to the possibility of negative biases.

Once we know about the the systemic bias, even if accumulated unintentionally - then what do we do about it. I think we are obligated morally and technically to do more after that point.

A similar situation exists with accommodation of people with disabilities. Once you know a building architecture can shut out people by design decisions- at some point we worked through at least some basic level of agreement on principle that accommodation is required.

That was my point. If the training data contained mostly white faces and few black faces, the AI would have fewer cases to base it’s weightings when trying to analyze a photo with black faces.

I'm sorry but no. I don't believe this at all. What are the chances of say, you finding a dataset maliciously labelled by racists and how would you uncover that? Would you ask every software engineer how he felt about Black Lives Matter as he tagged images? It's more likely that there is malicious intent in a company where 2% of software engineers are African-American than not.

Reply on news.ycombinator.com