what's this enormous risk they're talking about? racial bias in x-ray reading? race can be a risk factor in plenty of diseases, why should we actively try to remove this information from medical images?
Because in the US some people have a hard time understanding that all races and genders deserve to be treated equally as humans with the same access to goods and services. Further, that there are disparities in care based on race/ethnicity[1][2] and gender[3][4] because of that racism/sexism present in the systems. This then leads to requiring that race/ethnicity and gender data be scrubbed sometimes to keep people from impacting outcomes based on their own biases.
[1] https://www.americanbar.org/groups/crsj/publications/human_r...
[2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1924616/
[3] https://www.americashealthrankings.org/learn/reports/2019-se...
For example, not having race data on resumes is generally productive, because that categorization can't provide a meaningful input to the decision associated with an individual person. Even if it were to be the case that there was some correlation between race and skill at whatever job you're interviewing for[1], the size of the effect is almost certainly small, and in the meanwhile you've also controlled for any bias in the person doing the reviewing.
If you're having a machine look at a dataset, and the machine determines that race or ethnicity is a material factor in determining some attribute in that dataset, you're not doing anybody any good by denying that fact and destroying the result.
[1]Let's ignore for the purposes of this discussion, fields (like certain sports) where extreme competition combines with a position heavily dependent upon racially-linked physical characteristics. Though even in this case, there is still a (different, weaker) argument for suppressing race data in "resumes" (yes, I know, ballplayers don't submit resumes to their local NBA franchise)
I think the trickiness is in providing the machine unbiased data to begin with so that it doesn't incorrect associations between features like race. The most egregious examples I'm aware of are the machine learning systems used to suggest criminal sentencing, but, apropos to this topic I believe there are cases where it may produce erroneous associations in something like skin cancer risk.
If the outcome that you're trying to predict is also affected by perceptions of race, you've built a gossip feedback loop.
Now look just above that latter table, showing distances between East Asians and Europeans. The distances are far greater--more than 10x.
The precision with which we can identify and track ancestry, often based on small fractions of DNA (Y-chromosome in particular wrt Ashkenazi Jews, not mtDNA as one might think) doesn't imply the degree of genetic distinctiveness.
Then you are not pretending very well. When I lived in the US I was shocked at how often it was an issue. It permeates nearly every aspect of US culture.
The icing on that cake: A government-run interactive map so you can lookup which races live in which neighborhoods. Some versions allow you to zoom in to see little dots representing clusters of black or white residents. https://www.census.gov/library/visualizations/2021/geo/demog...
https://www.healthit.gov/isa/uscdi-data-class/patient-demogr...
https://dicom.innolitics.com/ciods/procedure-log/patient/001...
If interested, searching for "dicom conformance" should yield lots of docs that probably contain specific values for those things.
One common issue is a lot of these kinds of tags rely on optional human input and are inconsistently applied. As opposed to say, modality specific parameters produced by a machine, which are consistent.
DICOM is a great example of design by committee, with the +'ve and -'ves that implies.
no, it implies there is a signal in the dataset that could be something other than clinical. This means that until they can pinpoint the cause, or the thing the AI is detecting, all the other things it predicts are suspect.
ie if the AI thinks the subject is west african, then it might be more inclined to diagnose something related to sickle cell.
Or north western european woman in her mid 60s vs a japanese woman might get widly different bone density readings for the same level of "blob" (most medical imaging is divining the meaning of blobs and smears )
Similarly, if the train/test sets used here - for X-ray based diagnostics - using Machine Learning relies only on specific races, then the performance might be worse for other races, given that there's a new discriminatory variable in play.
The obvious solution here is to reduce bias by ensuring race is part of the dataset used for training and testing. Which, due to PII laws in play, may actually be quite challenging! Fascinating tradeoff imo.
Suppose AI #1 got a higher score on the training data and AI #2 had a more accurate diagnosis. Obviously you want #2 but if there is bias in the training data based on race and the AI has access to race then eventually you overfit into #1.
A human doctor is also a black box, in meat form.