ChatGPT's language model doesn't remember chatting with you. It doesn't remember anything not in the training set. It's stateless.
It only seems to remember you because the chat transcript gets fed back in again. No "game" is running, most of the time. There's no other "game state."
There is a writer. It can continue stories with convincing fictional characters. But the fictional characters consist entirely of words. This is the same sort of existence as the fictional characters in a novel.
The helpful AI assistant itself is a fictional character created by the writer. Each time you start a chat session, you create another clone.
If you wanted to keep a story going, there are other writers. The writer you're using could be replaced. It should make upgrades easy as technology improves.
Looks like we are going to end up chatting with increasingly intelligent fictional characters who don't actually exist. What sort of meme-creatures consist entirely of words?
No hay banda. There is no band. Il n'est pas de orchestra. This is all a tape recording. No hay banda and yet, we hear them. If we want to hear a clarinet. Listen... Un trombon à coulisse. Un trombon con sordina. Sient le son del trombon in sourdine. Sient le son! A muted trumpet. It's all recorded. No hay banda! It's all a tape. Il n'est pas de orchestra. It is an illusion. Listen!
I understand that it's just predicting text and the model is only working when it receives input, but what if someone took the LLM and made some kind of cognitive loop in the background where the same LLM reads information, thinks to itself when you are gone, and stores these things in its chat content then tells you about the cool stuff it read about the next day. It may be just text, but now it's "thinking". At what point should we start to consider it as a creature with rights?
Maybe it's a text in combination with a writer. But you can also swap out writers, so there's a ship of Theseus issue.
This ends up being much like the argument about whether a virus is alive.
Or, to simplify...why is a dog "alive" but a chatbot with a cognition loop is not?
As a non-expert in the field I was hesitant at the time to disagree with the legions of experts who denounced said dude and his claims. I know enough to know, though, that there is a longstanding tradition/bad habit of advances being dismissed by those in the field itself as "not real AI". Anyone, expert or not, in 1950, 1960, or even 1970 who was told that before the turn of the century a computer would defeat the world chess champion would conclude that said feat must have come as part of a breakthrough in AGI. Same if told that by 2015 many people would have in their homes, and carry around in their pockets, devices that can respond to spoken queries on a variety of topics.
To put another way, I was hesitant to be as self-assuredly certain about how to define consciousness, intelligence, and sentience—and what it takes for them to emerge—as the experts who denounced the ex-Google AI guy. The recent GPT breakthroughs have made me more so.
You may find a recent Sabine Hossenfelder video interesting. <https://www.youtube.com/watch?v=cP5zGh2fui0>
If you had an LLM that had unlimited chat storage, and gave it multiple containers for things like memories, idle thoughts, daydreams, aspirations, etc, and you had a loop for each of those where it reads information and reasons through it to form those thoughts, then stores those thoughts in certain containers and brings it up in conversation...would you feel bad if you called it a name and it reacted with emotional text? It's still "just an llm" with a bunch of overhead.
A recent Reddit post discussed something positive about Texas. The replies? Hundreds, maybe thousands, of comments by Redditors, all with no more content than some sneering variant of "Fix your electrical grid first", referring to the harsh winter storm of two years ago that knocked out power to much of the state. It was something to see.
If we can dismiss GPT as "just autocomplete", I can dismiss all those Redditors in the same way.