Today, depending on which model you use. You’re making unstated assumptions. And that’s not a fundamental property of LLMs, it’s happenstance. LLMs are capable of ‘plasticity’, by design.
It is possible to train on every prompt. ChatGPT is not currently doing that, but it is in fact possible to do.