Language is the I/O of the network, not the network itself. A human doesn't stop feeling pain just because we locked them in a box and reduced their interaction with the outside world to teletype. We simply don't know what's going on in the AI box.
> Maybe there's some future embodied AI
That changes nothing about the network itself. That just changes your perception of the network into something more familiar.
Most wouldn't readily accept that a Claude instance or a mysterious teletype smells lavendar just because it output a text that claimed to, even if it characterized the smell of lavendar through beautiful and relatable prose. Only if we had some independent reason to believe that it was capable of smell would we consider those words to be true testimony to the experience.
Pain, as widely used in matters of ethics, is in that class. We traditionally recognize it in some living bodies (only a rare few capable of language) and accept that, as living bodies, humans who communicate about it with seeming sincerity are very likely to be experiencing it. Although even then, we may remain suspect if their words are our only source of knowledge or if we have some deeper intuition about why those might not reflect a real happening.
And those are like vision, which LLMs are pretty good at. It's all just sensory information, 1s and 0s going into the network, which in turn change the behavior of the network in some way. Furthermore LLMs have demonstrated some form of vision even before we started training them on actual images, as could be seen by the crude vector graphics they were able to produce.
That single component of vision that you're talking about enables LLM's describe visual content, but in living creatures vision is a much more comlplex and multi-component system that can elicit non-verbal reactions like gaze aversion; cardiac or salivary or libidinous modulation; seizures and migraines; skin changes; etc
It's completely reasonable to believe that bioligical sense systems like vision or smell or taste or pain can be manifest in electronic systems. But one channel of "1 and 0s" being fed into even the most sophisticated language program isn't going to approach parity.
Like I said before, it's not unreasonable at all to have a discussion about analogs to more predominanently verbal faculties becoming manifest in these language systems, but there's a very profound and inescable reductivism being applied when you start to reduce environmental senses like pain or vision or touch to AI system as they stand today.
You can get away with that reductivism as a communicative metaphor for the sake of framing certain research endeavors, but you need to keep a handle on how thin that metaphor ultimately is. Anthropic is embarassingly/sophomorically eager to let go of it.