It sounds like a way to set up a positive feedback loop with terrible consequences to me. I don’t know enough about ML to be sure, but I know GIGO. If you feed bias into a model then let the model guide decision making, that seems obviously flawed. You bias the model not to give loans or credit to black people, reinforcing their lack of credit, which leads to the model confirming its bias. Maybe you can claim it’s a socioeconomic bias and not racist, and in some countries it would be true, but not in America. How long ago was it that black people had to fight past mobs to integrate schools? How long since redlining? If socioeconomic are tied to a history of oppression based on race, then ignoring half of that equation isn’t honest. Correlation is not causation, but does “AI” know that?