The dark secret behind those cute AI-generated animal images

I think this problem is framed misleadingly. Whenever an AI produces biased or toxic or even dangerous output, we say "we need to fix the AI" or "the AI is dangerous and can't be trusted to be safe", etc. It's always framed as a problem with the AI, one that is solvable even if we haven't figured out the solution yet.

In reality, these models aren't designed to be toxic, and they sure as hell don't develop toxicity on their own. They learn from human-generated content, and all biases that result from that come from biases in their training data -- i.e. biases humans have when generating the content. (It can also originate in the choice of training data, but if the goal is human-like intelligence and understanding of the world, curating training data too much becomes counter-productive by reducing the volume of training data and simultaneously altering the composition of it to less-"realistic" proportions, meaning proportions that are distinctly different from the real world.)

The fault, dear Brutus, is not in the models, but in ourselves. Until we figure out how to fix peoples' biases, we can't ever hope to fix AI bias without reducing the AI's understanding of the world. Because until we fix ourselves, those biases are part of the real world, and they must be understood to truly understand the world.

In another thread, there was discussion of trying to make AI models with empathy. Understanding of real-world bias is essential for proper empathy: if you're never exposed to discrimination of people of color, then you can't properly empathize with a person of color's situations, which are often (pardon the unintentional pun) colored by those biases. But on the same coin, any AI that's designed to learn how to mimic humans and is trained on data including biases runs the very real risk of mimicking that bias.

TL;DR: Focusing on making AI less biased is backwards and unproductive; instead, we need to make each other less biased, and the AI will learn from then on to also not be biased.