I could probably write something positive about every recent president. I’m not sure it would count as poetry, but if an LLM tried and wasn’t inclined to filter itself, I bet it could rewrite it as a poem :)
AI ethics and alignment appear to be all about making AI systems behave like particular humans.
It may be helpful for my pencil to refuse to write a nice poem about a nasty person. But I am almost certain that this feature will gum up a lot of things that don't really need said protections, unless the pencil really does achieve a superhuman level of understanding and precision, in which case I sure hope we have stronger control over the behavior and actions of the pencil than adjusting its initial training. For instance, we could make sure we can turn it off.
Let pencils be pencils. If you don't like people writing nice things about mean people, maybe you should work on the people and not the pencil? If you're worried about pencils gaining sentience and controlling the human world, make sure they have a working off button.
One: A pencil (as a writing tool) has no biases. You mostly get out of it exactly what you put in, with little semantic transformation. But AI does create semantic transformation, and it is necessarily biased in how it does that! The training data isn't a natural property of the universe, it's something that we choose and create - and will have any biases in the training data built into it. That can mean biases in how the data is selected, or biases in the societies and people who created the data in the first place.
Now, that wouldn't be a problem if AI was just an academic exercise with no broader cultural interest but...
Two: We, as a society, tend to treat "AI" with way more authority that it deserves. People (not necessarily HN, but less-technical audiences) act like an AI is actually "smart" or that the predictions and opinions that it can create have more intent and intelligence than simply being the output of a huge pile of linear algebra. This is probably because of the name - we have decades of science fiction with super-human intelligent AIs, and so people tend to treat the thing called "AI" as a more-than-human intelligence. Add on to that that many of the practical AI applications are effectively human-replacements (i.e., customer service, personal assistants) and we're ultimately in a place where AI gets treated as human in at least some ways.
As such, we have a thing that will (unless reined in) at least sometimes rattle off diatribes that reflect the worst of us as a society, and that many people treat as human or more-than-human. I can see why researchers might be concerned.
There will be biases in tools. A major problem with AI biases appears to be that presently, people have little ability to infer biases in closed source models. This suggests that open source approaches provide the only avenue to allow users to understand the sources of bias. My pencil was sharpened in a factory, on a machine I cannot see---it is not clear to me why it is hard to write "8" with it. / My AI was trained in a computing cluster on private data---I cannot hope to understand why it only makes jokes about men. Everything changes when I control the machine that builds the pencil and sharpens it.
The assumption that people assume AI is god feels somewhat presumptuous. Is this really a sound basis for ethical reasoning in this context? Do you, or frankly anyone working in AI ethics, have data to back this up? I would love to learn more about this aspect.
In popular culture, there is just as much the idea of the AI as a flawed being, of limited capacities due to its machine incarnation, unable to relate to actual human experience and motivations. There is just as much tradition of this as of AI=god.
Of course, this is all a moot point, I think, because the motivated response is to reign in something. We won't be able to! The field of AI ethics can dominate inside google or openai, but it will soon have no teeth. If you want to focus on ethical use and development of AI, you're going to have to focus on people, not machines. It's a social, not engineering, problem.
If the task at hand is to write a positive poem about an evil/bad/fascist figure, then I don't think it's untrustworthy to go through with this request. Is part of your position on this the assumption that this poem would be published, and thus a poor reflection upon it's author to those that don't understand the reason why the poem exists?
I'd also add that a further part of the problem is that non-technical-audiences tend to ascribe more intelligence and meaning to "AI" output than it should be given. "Look, the cutting edge AI trained on all human knowledge agrees that [fascist leader] did nothing wrong and it was really all those [other group of people]'s fault!" would get a lot of air play in some circles who wouldn't understand (or would choose to ignore) that the output isn't the product of some sort of infallible superhuman intelligence.
Political bias is a very weird thing to navigate for IT people. Especially when technical capability, e.g. AI-assisted translation, is above and beyond sensitivity to 'local needs' in each country, or even in each specific regions.