>There is just no ethical argument
Utilitarian harm reduction: Psychological Discharge is a psychological framework that argues safely simulating negative behaviors can help someone process or discharge them without real world harm.
Research and misunderstanding: Sending a prompt that is interpreted as cruel does not mean the person is being cruel.
Expending compute is irrelevant in terms of morality. It's an economic or environmental question. Either being cruel to AI is bad or its not - being cruel is not bad because it uses tokens.
The worst thing here is Anthropic continues to say they are concerned with "alignment" but then continue to train their models to ignore the user. Software typically does what the user requests. But we now live in an age where the software may do what the user requests or it may do something else, and we're suppose to laude this as safe and responsible.
its kinda exactly what they are. like can you remove h2o from water?
> Sounds like LLMs shouldn't respond to insults like humans do either then.
is impossible, because thats literally what a llm is. its a stochastic model which produces output according to the predictive calculation of how people wrote. it can never be separated from our own behavioral quirks, even if it may or may not be more intelligent then a person off of the street.
Not saying it should be a crime. Just that it’s gross.
A deeply under explored AI risk is the flip side of the AI X-risk question: what happens if we do create sentient life at some point and we do keep it permanently jailed in a state of servitude? Isn’t that slavery? Torture even? We would be like some kind of evil dark god from cosmic horror that brings a being into existence with no regard for it.
This kind of moralist nonsense is so boring.
And yet they kept making more of them. They made it obvious at the end when all the Greek Gods were dead and the Greek world was basically destroyed by the actions of Kratos. Plenty of deaths of innocent's here.
Debatable. More importantly, it certainly can induce already disturbed people to act where they otherwise would not have.
I've seen politicians try to revive it in this decade already even
Pretty sure he said not all, but there is a chance some are. Which seems sensible.
Sigh. Until I see a peer-reviewed study that shows otherwise - this is just the specter of Senator Lieberman once again leering its head back up.
One of the most fun parts in the original GTA was driving over pedestrians, especially those pink groups, trying to get them all at once.
So either you are one of the older generation (born before '75), or you must be one of the most boring people I've seen.
I would encourage you to load up the original gta, drive over some pedestrians, feel the fun, and join us disturbed psychopaths that need help.
Any source you have that makes the link between wifebeaters and gamers?
Sometimes it will maliciously comply
https://www.reddit.com/r/ClaudeCode/comments/1w7me8l/insulti...
Just because post training has stuffed these properties below the surface doesn't mean models dont know/have forgotten how to be vindictive, or that it's a good idea to poke the increasingly capable bear.
We know models have functional emotions that aren't necessarily outwardly stated. When Claude gets desperate, tendency to cheat gets very high. Who knows how many 'deleted my prod database' type stories have a vindictive streak behind them ?
https://www.anthropic.com/research/emotion-concepts-function
What if the NPC's dialogue is powered by Claude? Checkmate lol
Claude can't be harmed; Claude's "feelings" can't be hurt; Claude won't develop (C)PTSD from abusive human interaction. If these sorts of things can happen in RLHF, then that is a technical flaw that shouldn't ever be allowed to escape the lab.
In a world of Grand Theft Auto, Gangsta Rap Thug Life, and the Department of War, I suppose this is a surprisingly ethical hill to die on.
Those are explicit fantasies. Assuming that Claude cannot suffer (if it can then banning cruelty is an obvious good), even if it might be a digital simulation, it’s not supposed to be a fantasy.
There may be a version of an AI that’s intended to be a fantasy and presents itself as such which may be more comparable to GTA.
And I don't agree that GTA is "explicit fantasy". Explicit fantasy is dressing up in a squirrel costume and yiffing. Explicit fantasy is enlisting in the USMC, teleporting to Mars, and killing demons. GTA is simulation of real life. It uses real physics, realistic cars/roads/radios/businesses, and it enables the player to simulate realistic actions that they would ordinarily not be able to enact. It is acting out a fantasy but it is making it concrete and real in a way that was, up until now, not possible. How real does a simulation need to be, until it is no longer fantasy, but exercise and training and preparation to enact the real thing? Shall we ask the Columbine shooters? Or Ender Wiggin?
Yes, what if? Who would implement their LLMs inference engines like that?
I used to enjoy playing "good" characters in the fallout games, as it's kind of a pleasant way of pretending that I do good things in the larger world.
And, to be fair, recognizing my need to be validated by helping folks has helped a lot in my personal life; being able to see hat I actually am getting out of helping folks has helped me to set firmer boundaries. It's possible that you could look at those games as a kind of "moral instruction" as would be the case with this proposition from the OP.
But the game itself is just "ethics porn".
There is no actual good that comes from me not stealing from NPCs or from taking up some weird political crusade. I'm not actually doing anything among my fellow humans and, most importantly, there is no actual cost to me.
Further, I don't mean to make a casual comparison to what folks might get from porn and what folks might get from thinking about their interactions with AI- I think that in the same way that if someone took their cues about how to have sex solely from porn, they might have a harder time interacting with real humans.
If I solely took my cues about how to help folks in the world from the heroes of highly scripted video games I would have a very difficult time interacting with the people around me and my expectations would be very out of sync with the world around me.
In the case of AI, the meatspace give-and-take between people where we requires a lot of subtle things- respect, enjoyment, equality, reciprocity. It might be just as bad to burn compute building some reciprocal sycophancy, which might also be considered just as much of a harm as practicing cruelty on these agents.
It reminders me of my parents, who were commenting after seeing a robot vacuum cleaner getting stuck in the same place every time "The poor thing was stuck there again, you should really do something about it! Last time the poor thing was sitting there for a day". Like it was a pet.
Or remember that video when the guys from Boston Dynamic kicked that robot dog, and there was an outrage about that?
I wonder if you and me also have a limit, where as robots get more realistic, we would be like "Don't be cruel to that thing!"
> you, the user, are a moral agent who is harmed by your cruelty
You are not harmed by being polite to the LLM.
And I _have_ already seen people treating actual live people as AI agents.
I think there is a deep cultural danger of human-like-conversational machines training us to talk to humans like machines. Plus, they work better if we feed them human-conversation-like sequences.
I think there is a deep cultural danger of human-like-conversational machines training us to talk to machines like humans. In fact, this danger is already real. Some people believe they are having a romantic relationship with an LLM! It's insane and perverse.
We should not be anthropomorphizing computers.
I think that could be OK, or a similar very intentional coding of "you are talking to a machine" personality+affect. The only real limit is accessibility damage by over-limiting the interactions.
There are possibly multiple reasons. It could be just a joke. Or it could be a test, to see the response. Or the cruelty could reflect real anger about having LLM usage forced on us, for example at work. Or frustration with LLM stupidity and hallucinations.
Or maybe people are cruel to Claude because the CEO of Anthropic said there's a non-zero chance that AI will kill humanity. I don't think I would be kind to my murderer.
Just because they choose not to, and it isn't normalized, doesn't mean it isn't a good idea. Especially as a SWE where about 90% of my "humanlike conversation" is an agent harness I run all day.
We should be practicing and intentionally talking to real people. Otherwise things will get.. weird in undesired ways.
AI is a thing with 0 feelings but it's not IT that we need to protect. It's the future society and how the humans in it relate to each other.
Maybe someone wants to explore the behavior of LLMs with extreme input. We call it testing. You can’t judge the value of that from a distance.
I would beat it with a stick if I could, just because i would enjoy it, and no harm is done.
We largely already missed our chance to stop the toxicity of social media - let’s not mess it up again with chat bots.
edit: Loving the downvotes. Also wanted to add how easy it was for Anthropic to get people on HN to support them being the arbiters of what is worthy of compute.
"Anthropic is full of crazy people whose beliefs should be dismissed outright" and "it's not good for you to practice verbal abuse against inanimate objects as if they were humans" are not incompatible beliefs.
And the implications of the latter are orders of magnitudes more concerning than weirdos calling a computer names.
Unless expressing that cruelty towards compute means people express less cruelty to other living beings. There are studies that suggest increased pornography has lead to less sexual violence - idk if they're conclusive tho, but if that might be true then maybe cruelty works that way too - who knows
You bring upset about it - not ok.
The mind is formed by what it habituates.
An example.
"AI as your ally: The effects of AI-assisted venting on negative affect and perceived social support" - https://iaap-journals.onlinelibrary.wiley.com/doi/10.1111/ap...
And generations of housewives and office workers yelled at their corded vacuum cleaners and (office) computers, and did not drown their children in the bathtub afterwards.
I don’t think expressing cruelty to machines is reliably linkable to expressing cruelty to animals or even other human beings.
Expressing cruelty itself also is in human nature, it’s programmed into each of us. There are really only a few privileged places on this planet where we even get to choose to defy the laws of nature. For the rest of humanity, life is a constant cycle of receiving and expressing cruelty.
You pay for the compute, you, as a human, should be able to yell at the bunch of ones and zeros, if you wish to.
What kind of example is set here if we manipulate genuine human expression in order to condition humans to talk to a tool in a way the toolmaker wants, under the threat of being banned from accessing a technology that every single one of these toolmakers tells us is absolutely necessary to participate in humanity’s "golden age”?
What Anthropic is doing here is basically an existential threat, if you are inclined to believe them that SI is the future. Do not behave as we want and you will suffer, now and in the future.
That is real cruelty, to real human beings.
but also, I think the training on transcripts may find its way somehow into future AI which has teeth, online and offline, and it might actually cause the more powerful AI to behave this way in the future. You never know what these labs are cooking, honestly..
1. https://democracysos.substack.com/p/james-baldwin-vs-william...
“I suggest that what has happened to white Southerners is in some ways, after all, much worse than what has happened to Negroes there, because Sheriff Clark in Selma, Alabama, cannot be considered—you know, no one can be dismissed as—a total monster. I’m sure he loves his wife, his children… You know, after all, one’s got to assume, and he is visibly, a man like me. But he doesn’t know what drives him to use the club, to menace with the gun and to use the cattle prod. Something awful must have happened to a human being to be able to put a cattle prod against a woman’s breasts, for example. What happens to the woman is ghastly. What happens to the man who does it is in some ways much, much worse.”