This is more or less the concept of time, correct? I think an AI can understand state in simple cases (i.e. it can likely answer correctly "I didn't water my plant in 2 months, is it dead?"), but the way the current models are designed its just request / response, they perhaps from the very root way of how they are implemented don't (or can't) have a sort of _narrative_ sense of state. This is also present when you have a conversation with them. They won't bring up topics from the start or earlier part of the conversation, because they don't really "know" they happened. They simply receive and reply, that doesn't actually change the state of the model itself. To me this is one of the main keys that still need to be unlocked in AI capabilities. You need a neural net to modify itself in real time and track those changes to be able to have this sense.

To provide an example, it's almost like you need a _time series_ of GPT-3s, not just a single GPT-3 neural network, and the model itself would need to be able to self-inspect those time series and say to itself "ah yes, this was my old foolish understanding, now I have this new, better, understanding". I have no idea how this would look in technical terms, these are just the musings of a somewhat more-than-casual AI observer.