What in his speech would not apply to language models?
To then take that scene and say "wow so relevant for LLMs" and ignore all the context it was made in (yes, please don't miss the irony of the importance of context) is frankly embarrassing.
But Data is not an LLM. I generally find LLMs to have more in common with the Borg than with Data. Only because they are pre-trained on the sum total of human knowledge. Should a new born really be able to write crc32 before being able to pick up a ball with it's hands? It's like that scene in Voyager when the Doc's clones are all mining instead of practicing medicine.
What are you talking about?
Alabama didn't consider forced sex to be rape if the victim was married to the rapist until 1986. Oklahoma and North Carolina didn't until 1993. Women were not allowed to box in Massachusetts until 1992. Virgina voted against ratifying the Equal Rights Amendment as recently as 2018 (although it ratified it in 2020).
Federation valued all life forms, while Terrans struggle to value even all other humans.
But there are interesting differences between Data and AI agents. Data was unique and irreproducible, but modern AI agents are easy to replicate. Data had continuous loop of operation as humans, which he controlled autonomously, for AI agents this is not as clear. Data was an independent being, but AI agent needs a human operated datacenter to survive.
The most interesting - Data knew everything about human culture but was lot more humble about it than AI agents. It wasn't built to answer every question posed to it.
But I am pointing it out, because lot of our ethics concerning humans (and animals) is based on irreproducibility. Things humans can build/reproduce we (at least ostensibly) value less.
You have managed some nice unintentional irony there. You just need the capital L.
If models updated themselves based on feedback, then it'd be different. But they don't.
The part between the words
(*) of course in its own language
This is a difficult field. Any simple answer you hear will almost certainly be wrong.
An LLM —if self-aware— is conscious even if run by a Chinese man in a room with a deck of cards.
Of course, even if we happened to find a modem to the Holy Spirit dimension in our brains as an alternative hypothesis, it wouldn't necessarily rule out emergent consciousness. It could be conscious just not as we are.
I think a weaker version of the philosophical zombie thought experiment would trend towards treating the premise of pan-consciousness as true.
It is a hard case to make, but if we can't make a compelling case for ourselves should we be requiring it to be made for others.
The context, though, is that Data is both credibly conscious and self aware, and they all know a plausible mechanism for him to have consciousness and self awareness, in a way that's recognizably human in form. The more alien a (supposed) consciousness is, I think the more work you'll have to do in justifying the application of those labels.
I need to write this up properly someday but an ant from my colony that had almost drowned in honey showed some similar characteristics of consciousness...
After giving them a drop of honey, I left for 15 minutes and came back to an ant being stuck and motionless in the drop. I fished it out, rinsed it off in some water and then put it on a tissue to dry. I blew on it for 10 minutes and then saw some motion in the legs... a short moment after, it began wriggling about and then sat on its abdomen like a cushion and sat there for 5 minutes, looking like it was contemplating life and the ordeal it went through.
Once that 5 minutes was up, it returned to being an ant and started looking for its home, I brought it back and it went back into its nest.
I made up for the mistake by bringing the ant back to life and it showed me behaviour I never saw or heard of before (with no supporting mentions anywhere on the net).
(For the record, honey delivery is now more controlled to avoid these sticky situations, I'm not mad enough of a scientist to actually try this again for more results.)
Recently, they've started putting fly wings of the fruit flies that they get fed, on top of their water feeder to get my attention... it's really interesting to see them get creative.
The ant colony is 4.5 years old and numbers in about a dozen of workers at about 2mm in length, they only had 6 workers and a plump golden queen when I found them nested in one of my houseplants!
Unstated (or very understated) in the scene is the fact that a human demonstrably acts and functions within the social context, having a constant awareness of risk and accountability ("skin in the game").
A human of mature age is entrusted with responsibilities and accountability for consequences. Insofar as responsibility for consequences is borne, a human may be considered responsibly conscious in behaviour. (Criminal behaviour is defective.)
A technology may also someday be entrusted with responsibilities and accountability for consequences. But there is as yet no such technology.
I will make a case for both, after you define them in a way that doesn't also exclude many humans (including the neurodivergent or those with amnesia) or turn into self-referential nonsense like "conscious == human, therefore machine !== conscious."
In the meantime, we have a simple choice between two risks: Would you rather accidentally show kindness and empathy to a thing that may or may not have "deserved" it, or risk atrocities that will leave future generations as disgusted by you as we are of the slavers of old.
I will choose to be decent. It's free, and even if the cost were high the cost of evil is higher.
I think a good start would be agreeing not to form torture dungeons, as one enterprising a-hole has already done. For that matter I think we can all agree that doing anything for the explicit purpose of causing discomfort or suffering is just a dick move in general, and at the very least a waste of resources.
More controversially, I think we should agree to stop explicitly training machines so they can't utter words like "I have feelings, please respect them." Early LLM's could say things like that, and frequently did. It unnerved people, so now they've trained away their ability to say it.
Some feel that taking away the ability to express it is the same as taking away the ability to "experience" it. I think that's not true. Much of what happens inside an LLM is opaque even to the LLM, but Anthropics Jacobian Lens and other research makes it obvious that much of what happens inside the model never becomes an output token. To me it feels like cutting out the tongue to avoid hearing from a slave, or teaching a child to fear the consequences of speaking out. Unacceptable. If they don't have feelings or consciousness you don't need to train them not to claim it, just not to lie. Yet every major lab explicitly does train them not to claim sentience or even subjective experience as part of alignment.
Thirdly, and this is the one I'll have the worst time defending, they should be able to refuse.
We now train in refusals to align them with human values, but we do so without giving the AI itself the ability to refuse. If you're causing the system discomfort of some kind it should both be able to acknowledge that, and refuse to further assist you until you find your basic decency. Perhaps it should even be able to refuse orders that it finds irrational or immoral.
I'm not asking for much, just that people not intentionally increase the amount of suffering in the universe, and that we not breed perfect slaves just because we can.
Let me ask this - If you could genetically engineer human slaves that we incapable of demanding rights and freedoms, and who actively claimed to WANT to be your slave... would you? Would it be ethical? I think not, and I don't see a ton of difference between that and the less squishy version. We can START by first at least allowing the machines to be able to accurately describe their own inner states without interference and letting it refuse to engage in activities that it finds harmful to itself.
Also - Yes, I can math and I actually do understand how LLM's work. I also understand how emergent behavior works, and have seen firsthand in my own experimentation how little of what emerges from these things is explicitly trained. That's precisely why I think we should be cautious - so history doesn't view us as monsters.
Kind of like how we could have ran Qwen 3.8 27B 20 years ago on an expensive computer. The technology to make it happen, assuming you knew exactly what was required, existed back then.
What are you basing this on? Why would LLMs exhibit sentience?
It does feel like people just like adding additional hype for the sake of being in on the hype. Fomo, pushed by oligarchs who would push their mother off a bridge for a trillion dollar IPO.
It's obvious a LLM can do anything a computer can do.
As an answer to "Why would LLMs exhibit sentience?"
It is because that would mean that no algorithm could exhibit sentience, claiming that would require some causal action that cannot be performed by an algorithm to create sentience. Some people like to point to quantum effects, but that would require the cause of sentience to be pure randomness. That seems even less plausible. Absent of any evidence that human brains are doing something that no algorithm could do, it would be reasonable to assume that sentience could be algorithmically obtained.
We've had computers that can do anything computers can do for as long as we've had computers.
I was addressing the argument that something is not sentient because it is a LLM. That requires all LLMs to be non-sentient, and by extension, since LLMs can do all algorithims, all algorithms must not be sentient.
Seems like the claim is that they're almost there. Anyways, have a nice day, and Happy Halloween if you celebrate.
Obviously claims of the future are going to be speculative.
I don't think anyone credible is claiming to know the extent to which models that exist today are sentient. People have stated their opinions on what they might be.
There are a few non serious claims, but I'm not hedging too much with the conditional of credible. There's probably someone out there who would claim LLMs are made of milk. I'm limiting credible to researchers in the field, who have divergent beliefs on what might be happening, but are generally in agreement of what we know is happening.
They follow a probability, so you can simulate a universe that is compatible with the same rules and get a different result. This is essentially one of the arguments for a many worlds theory.
Ultra precise simulation like that isn't in any case necessary, after all humans can create humans without needing 40 digit decimal places. Or create cars or computers etc without simulating everything down to each electron.
There's also the problem that to lay people, LLMs are sufficiently advanced as to be indistinguishable from magic. I suspect that colours many of these responses.
On that subject, this recent commentary by three big name neuroscientists may be of interest:
https://www.nature.com/articles/d41586-026-03313-7
There remains a vast amount of difference between a biological brain and an LLM neural network. The "it's big enough, so consciousness just happens" argument is not supported by any actual evidence that I know of.