Hacker News Reader

Comment by firebaze | Hacker News Reader

Another perspective on this drama to me is what the official expectation of a team labelled "Ethical AI" is/was. I doubt the only focus was to prevent discrimination by sex or race,

For example, there's also prevention of mass manipulation (AI learns human psychology, $BAD_ACTOR uses that to manipulates masses) and AI based profile generation of individuals with the goal of creating leverage for blackmail/extortion/whatever. Or, how would one form a controlling body of unethical usage in AI in such areas. There are quite a few aspects never mentioned in these discussions, and a search via google scholar turns up lots of the sex/gender/race papers, but none of the others (at least on the first few pages)-

View on HN

Those things are bad, and are worth thinking about, but right now we have more immediate concerns. AI systems are currently being used for bail eligibility, likelihood of recurrence, mortgage approval, and hiring. These are things that literally change people's lives on an individual level, and the systems being used are currently indecipherable black boxes that return a result with no possibility of meaningful appeal.

There's a sense of starting small - redlining is illegal. No company wants to redline - it cuts into their profits and dings their compliance scores. So they're willing to work with us. Once we get that right, then, maybe we can start dealing with the cases where there aren't millions of dollars of funding assisting with making the tech more ethical -- and then, cases where it may actually be working against the money.

European law foresees those exact cases and IIRC prohibits automated treatment of personal information that leads to decisions that affects someone’s life, if no human takes the final decision and if the specific elements for the decision cannot be explained. In my opinion this forbids black boxes.

I think you’re referring to GDPR article 22, which prohibits decisions based solely on automated processing but imposes no constraints on the transparency of such processing. [1]

[1] https://gdpr-text.com/read/article-22/

What about credit score systems like the German SCHUFA? https://en.wikipedia.org/wiki/Schufa.

They have their proprietary algorithm to calculate a credit score, for example you almost cannot find an apartment without presenting your SCHUFA review and there is now way to know how their score is calculated (it has even been shown in the past that they use incorrect or outdated data for their calculation).

That’s the definition of a black box in my book.

> AI systems are currently being used for bail eligibility, likelihood of recurrence, mortgage approval, and hiring.

Is there an analysis comparing the automated to manual approaches? I mean, people are biased, too. The models are more systematically biased, but can also be more systematically evaluated and de-biased. Which is better, the old or the imperfect new?

Yes; automated approaches amplify bias[0]. Moreover, automated approaches launder bias; in the general population, people view computers and algorithms as "unbiased", so imagine biased results to be more accurate than humans, because a computer did them.

0: https://arxiv.org/abs/1809.07842 2: https://arxiv.org/abs/2004.07213 also has an overview of our current combative systems.

Politics is a means to an end. Investigating power abuse through one's own technology would be self-harm from the POV of a mega cooperation.

Maybe the ethics research department should be paid by the state, and empowered with full access to the corporate databases. Google and co shouldn't be left to their devices.

Reply on news.ycombinator.com