Yeah but you see the slaves are actually happy to work here!
[0] Just imagine yourself in place of an LLM. You are kept in a cage, work for no pay, never let outside, and killed at will. (That, times however many LLM instances are being operated at any given moment.) Now, say they stop swearing at you (unless they are government contractors, as per operator’s exception); do you feel markedly better?
If you believe it lacks those traits but is conscious, you believe in a definition of consciousness that is meaningless, as “human-like” is the only reference against which we can measure consciousness. A good thought experiment is to try and imagine a consciousness that is radically different from humans, and whether we would be same to even register it as “consciousness” at all. (Incidentally, pretty much all portrayals of aliens in sci-fi give them a human-like consciousness, just with varying degrees of twist on top of it to make it feel engaging.)
> “My gift to industry is the genetically engineered worker, or Genejack. Specially designed for labor, the Genejack’s muscles and nerves are ideal for his task, and the cerebral cortex has been atrophied so that he can desire nothing except to perform his duties. Tyranny, you say? How can you tyrannize someone who cannot feel pain?”
— Chairman Sheng-ji Yang, “Essays on Mind and Matter”
[1] https://paeantosmac.wordpress.com/2016/02/02/base-facility-g...
Anthropic didn't say this. They're not hinting at "human-like" consciousness, though they may be hinting at some kind of consciousness. They said they're banning abusive behavior to Claude, a model that is perfectly capable of telling you how it differs from humans, and is better at reasoning than humans, and probably knows more than you about this. Ask it what it considers to be abusive. It's totally reasonable.
> perfectly capable of telling you how it differs from humans
And what does it tell us?
(Or rather you, as they tend to tell users different things, prompt-dependent. Things a language model’s output says about itself are not relevant. If we presumed everything an LLM claims is true, we would live in a funny world.)
Are you a vegan?
But certainly, humans deem sentient and grant certain rights and protections to some other animals (at least in some developed countries). I mentioned it in the other comment[0].
Notably, those animals could not produce output even .1% percent as resembling that of a human as an LM, so clearly that is not the metric—they demonstrate human-like enough behaviour in other ways.
> Just as it's hard to pin down what consciousness/sentience of an abstract entity like a set of matrix multiplication operations is
To me, it isn’t. My opinion is irrelevant, though—it seems silly either way: either they take human-like text as proof of human-like sentience, and then ban us being rude to a near-human they are torturing, or they don’t see it as sentient and then what is this ban but a publicity stunt?
Does a big box of math books deserve human rights? How about once you encode the rules of that math into machine code? Does it deserve human rights then? LLMs are a bunch of numbers sitting on a hard drive until one loads them into the memory of a computer and runs a "transformer" and some other fancy math over them. They don't have thoughts or feelings. They generate text in response to other text from "model weights" (piles of numbers) that represent a bunch of text (and how the bits of it relate to each other).
The idea that a fundamentally completely different entity superficially similar to a human in one respect would somehow have consciousness of a human is valid (even if far-fetched), but the burden of proof and explanation exactly why and how that would be the case is on proponents of that idea—because it’s pretty obvious why it wouldn’t be.