I agree, I'm certain that's possible. Humans aren't special. But I don't think that's what an LLM is, at all.
A LLM is not a brain. Brains are the product of selective pressure, LLMs are the product of a scanner, matrix algebra and a loss function.
A brain is successful if it keeps a body alive long enough to successfully breed (not even necessarily its own). To do that it has an almost innumerable collection of competing factors. It's deciding the next action.
An LLM is predicting the next token, often of a string of text that is the product of some brain. If so, it's an entirely conceptual account (words) of some brains' subjective experience that is (at least partially - eg emotion) completely untethered to the world.
These are very, very different things.
For example, why would we expect something whose entire selection pressure is "what happens next" to develop a sense of self?
There is no self, there is only the next token. An LLMs behaviour might make it LOOK like it's doing things that imply it has a self, but that's illusory - it's just imitating the average tokens that humans produce in its current state - because that's the very nature of what it is.