Hacker News Reader

>But false negatives are also very bad

The principles of the law makes it clear that it's far, far worse to have an innocent person in jail that a guilty person go free. Many legal principles revolve around this.

> The principles of the law makes it clear that it's far, far worse to have an innocent person in jail that a guilty person go free. Many legal principles revolve around this.

Yes, absolutely. But there are two issues in this context.

The first is that facial recognition isn't a conviction. Presumably if it identifies the wrong person then they'll have the opportunity to demonstrate who they are, e.g. by showing that the name on their ID is not the name of the suspect, before being convicted of the crime.

And the second is that ordinary application of that principle is incompatible with eliminating the disparity, because if you shift in favor of false negatives in general then it reduces the false positive rate for all races but the proportions stay the same. But if you shift it only for one race then you're only making the existing racial disparity in false negatives even worse, which is hardly fair to minority communities because it implies they'll have a higher proportion of criminals on the streets victimizing their communities.

That being said, in this context having a preference for false negatives over false positives in general probably is a good idea, and it does reduce the absolute (rather than proportional) racial disparity in false positives -- which is not nothing. But it's the proportional disparity that people are always complaining about.

That has exceedingly not been the case in recent time. Consider the existence of plea deals which punish people for insisting they be proven guilty in a court of law.

> But false negatives are also very bad

Are they? Isn't our entire justice system premised on the idea that false negatives are far more acceptable than false positives? There's a problem when your false negatives are systematically racially biased, but beyond that, false negatives seem like what you'd expect and desire from such a system.

Why not change the training data to reflect an equal amount of the population? Officially the US has six “races” according to the Census Bureau. So, another option is to make each race ~16% of the training data.

Some might object to the decreased accuracy this gives whites — but if we’re going to have any hope of preventing existing systemic inequalities from being encoded in AI systems, we’ll have to socially engineer the data sets.

>"Some might object to the decreased accuracy this gives whites — but if we’re going to have any hope of preventing existing systemic inequalities from being encoded in AI systems, we’ll have to socially engineer the data sets. "

So to fix systemic inequalities for some groups, we're going to increase systemic inequality for certain other groups?

Definitely a step in the wrong direction as it's just going to swing hatred back the other way. And N-levels later of this "fiddling" you'll end up in a spot where neither side can reconcile with the other because each side has legitimate reasons to feel slighted and no one can look past that "hatred". Just look at the mess that is currently in the middle-east after people "fiddled", and then "fiddled" again to try fix it.

> So to fix systemic inequalities for some groups, we're going to increase systemic inequality for certain other groups?

No; in order to accurately identify people through appearances, we'll try to sample evenly throughout the range of appearances and features instead of sampling based on proportions within the population of the people setting up the system.

You're can't be seriously making the case that white people are such an overwhelming portion of the world's population that not concentrating on them in your training data constitutes racial discrimination. Not only racial discrimination even, but is a attack that will inspire a "legitimate" counter-retaliation.

No that's not what I'm saying. If you look at the quote I mentioned, the person was suggesting we "socially engineer" data sets in order to "preventing existing systemic inequalities from being encoded in AI systems".

This has wider implications and tacit-meanings that I think I picked up on and definitely implies doing more than just having an equal sampling of each racial group. How that would be enacted, I don't know. The point is to not swing the pendulum the other way in order to "fix things" as it'll cause problems down the line, but to rather just "fix things" in the most non-intrusive and fair way.

I'm glad you feel this way. So do you agree that it is therefore unacceptable to use such a system even if one accepts the surveillance state that comes with them since it demonstrably disadvantages minorities and making them "fair" would only make things worse? Or is this concern more selective?

I'm pretty libertarian, so I lean to simple solutions and my view on "fairness" is very non-complicated. I.e. Yes there may be bias in this technology we use, however, I would say that the important part we should really be focusing on is "how" that tech gets used, not the fact that it isn't perfect in some "unfair" kind of way. All sorts of things around us are unfair to all sorts of sub-groups of people that I don't think we should be going down this path and mixing government intervention in order to resolve it.

Yes, it's kinda unfair on some level that the algorithms used don't work so well on women/blacks/children/elderly as well as "white middle-aged men", but it's not unfair in the "government should do something about it" kind of way just yet. If someone gets misidentified, that doesn't mean police should just go out and "arrest" them. Rather we add a person into the mix and have them talk to that person, make a final call. That is actionable counter-case data that we can use to resolve the bias in the training sets.

A lot of facial-recognition systems include human components in order to resolve errors. E.g. I've worked with facial-recognition software here in South Africa that was developed by Asian companies where the majority of their data-sets and tests were using Asian individuals, so one has to manage error risk using humans in the loop.

How are you managing human bias in this model?

Depends on what kind of bias you want/need to manage? Corruption/bribery is a big risk factor in SA so that is usually the one most people worry about, rather than racial bias.

One of the things you have to look at is "how" that bias may manifest and if bias can even be applied in any negative sense in the realm of control you give to the user. If you give someone a task where they are to compare two faces, and they don't like the demographic of that individual, I suppose they might pick incorrectly (because you only gave them true/false type options). But you can build in review processes, reporting, performance feedback, etc to mitigate that. You need not treat it as bias at all, merely bad performance/mistakes that need correcting.

Consider a hypothetical scenario in modern police work: First pass returns a false positive on suspect match, police act on the false positive in a way that results in jail or death due to racial bias. This rarely manifests as a performance issue with easy remediation.

I'm not sure adding more humans to the mix is a solution to solving bias problems because humans are unconsciously biased and may be perversely incentivized.

I don't understand? I just gave solutions above how adding humans in the mix solves accuracy and can most-likely preclude bias.

Because now the system is only going to perform at peak accuracy in some fantasy world split up into 6 races of equal part man, woman, child, and elderly. If you’re seeking a realistic model, you randomly sample actual data.

There are many problems with socially engineering data sets. For one we would need to achieve consensus on what constitutes an appropriately representative (i.e. socially just) data set. And then we need a feedback loop to make sure that our dynamically changing inputs don’t knock our meticulously tuned output out of whack. And we’d need to continually adjust our model based on our changing definition of social justice. Finally we need to find social engineers with credentials in building perfect societies (I most certainly don’t trust Joe PM) to perform all this work.

Sociologists have been arguing forever that humans are inherently racist. Maybe these “AI systems” are just proving their point. Perhaps this bias is more a reflection of our inability to maintain global context that is also maximally relevant within our local tribes. Maybe we should focus on acknowledging biases and working through them rather than trying to transcend into some unicorn world where everybody has all the context all the time and bias is impossible.

Anyway, I see your census and raise you an FBI: white people commit way more crimes in the US [1]. So if your goal is to reduce total harm then social engineering is not your solution. Perhaps more accurately identifying the most frequent perpetrators (in the case of the US, white people) is exactly how you’d end up tuning your system...

1: https://ucr.fbi.gov/crime-in-the-u.s/2010/crime-in-the-u.s.-...

This attitude is one of the many reasons why we can't have nice things. Instead of trying to elevate everyone to best possible standing you want to reduce all to the worst one.

> Why not change the training data to reflect an equal amount of the population? Officially the US has six “races” according to the Census Bureau. So, another option is to make each race ~16% of the training data.

The difficulty with this is that you commonly don't have any control over the training data. Someone else collected it and you can use it but you can't choose what's in it.

And in some cases you already have census-level data, which means you couldn't even increase the number of people with lower representation in the data set by collecting more data, because there are no such people not already in the data set. The only way to have equal representation would be to throw out data you do have on the groups with higher representation. Which is obviously objectionable because we need to be improving accuracy for the group with lower accuracy (if we can), not reducing accuracy for the group with greater accuracy just to make the numbers the same.

Increasing overall accuracy won't help if it's applied to all ethnicities. It has to be an increase only for the affected type of people. But if you're going to do that, you could also just artificially cripple the accuracy for the too-accurate groups. That sounds like the easiest solution, if it's actually a problem to be solved in the first place.

No, it would help a lot. Let's say you shrink mistakes from 10% and 20% to 1% and 2%. That means the gap in misidentification likelihood has shrunk from 10% to 1%.

Huh? Isn't that just clever accounting? 2% is still twice as much as 1%. If a black person and a white person each travel the same number of times, the black can still expect twice as many additional checks as the white, the same as before the improvement.

No, it’s the appropriate measure of ”unfairness.” If you designed a system where 5 black guys got misidentified but 1 white guy, out of the U.S. population, that would be far less unfair compared to 2%/1% despite the ratio being higher. And why did you pick that ratio to compare instead of 90%/80% vs. 99%/98%? The right means of comparison — for fairness — is the absolute difference in proportion.

I don’t mean to imply that’s the right metric at all, of course, because it specially privileges minorities. The proper measure should equally weight the algorithm’s performance on all people in the population, or at least weight them irrespective of race. That’s just not a measure of fairness between races.