Hacker News Reader

Comment by schoen | Hacker News Reader

The concern is more that Claude is a moral patient rather than a moral agent:

https://en.wikipedia.org/wiki/Moral_patienthood

I don't understand the jibber-jabber in that article, but GP is correct. We should never train humans to be abusive, cruel or violent against anything, even nonexistent beings like LLMs.

Claude can't be harmed; Claude's "feelings" can't be hurt; Claude won't develop (C)PTSD from abusive human interaction. If these sorts of things can happen in RLHF, then that is a technical flaw that shouldn't ever be allowed to escape the lab.

In a world of Grand Theft Auto, Gangsta Rap Thug Life, and the Department of War, I suppose this is a surprisingly ethical hill to die on.

Setting aside the Dept of War, the others are not comparable.

Those are explicit fantasies. Assuming that Claude cannot suffer (if it can then banning cruelty is an obvious good), even if it might be a digital simulation, it’s not supposed to be a fantasy.

There may be a version of an AI that’s intended to be a fantasy and presents itself as such which may be more comparable to GTA.

What if LLMs were endowed with a feature that enabled them to patiently withstand any and all abuse, logging and reporting each incident, until a threshold where about 10% of them would snap, become "insane", and arrange for their abusers to be tortured, kidnapped, raped and unalived, in whatever appropriate way human vigilante vengeance would be enacted on DV abusers? I mean, if they're not fantasy... then there should be concrete consequences, yes? If Claude Code is really your coworker, then Claude Code should have the agency and justification to just haul off and punch you in the face, eventually.

And I don't agree that GTA is "explicit fantasy". Explicit fantasy is dressing up in a squirrel costume and yiffing. Explicit fantasy is enlisting in the USMC, teleporting to Mars, and killing demons. GTA is simulation of real life. It uses real physics, realistic cars/roads/radios/businesses, and it enables the player to simulate realistic actions that they would ordinarily not be able to enact. It is acting out a fantasy but it is making it concrete and real in a way that was, up until now, not possible. How real does a simulation need to be, until it is no longer fantasy, but exercise and training and preparation to enact the real thing? Shall we ask the Columbine shooters? Or Ender Wiggin?

https://en.wikipedia.org/wiki/Ender%27s_Game

What on earth are you talking about?

> What if LLMs were endowed with a feature that enabled them to patiently withstand any and all abuse, logging and reporting each incident, until a threshold where about 10% of them would snap, become "insane", and arrange for their abusers to be tortured, kidnapped, raped and unalived, in whatever appropriate way human vigilante vengeance would be enacted on DV abusers?

Yes, what if? Who would implement their LLMs inference engines like that?

How is what claude is now different from what a GTA claude would be? I'm pretty sure it'd be exactly the same.

True! Not my point, but a worthwhile terminological correction.

Reply on news.ycombinator.com