It's more subtle than that : weights don't change, but the internal state does change at each new word (which allows it to perform few shots learning such as taking and following instructions given in a prompt). This internal state is defined implicitly by the weights that don't change applied to the prompt and generated up to now text (which does change).
This internal state is quite big ("dim of the features" times "number of layers" times "current length of text"), and represent an expended view of the relevant context of the conversation, that include things both low level features like what's the previous word, and higher level features like what's the tone and mood of the conversation and what's the direction this conversation is aiming at, so that you can predict what the next word should be accurately.
This allow to have "chain of thought" reasoning. This sequence of internal states maybe can be seen as a form of proto stream of consciousness, but this is a controversial opinion to hold.
Everything usually is not persisted, except for the conversation themselves that will almost certainly be used in subsequent training datasets where they will be used to improve further models.
To keep anthropomorphising, it's like you have plenty of independent chat sessions everyday where you are free to bounce thoughts as they spring into your mind. During the night, some external entity (which are often other models that have been trained to distinguish good from bad) evaluate which conversations were good and which conversations were bad, and make you dream and replay the conversation you had during the day and to updates your world model weights. And the next day you woke up with a tendency to produce better conversations, because of the conversation of yesterday.