The problem is swapping LLMs can require rework of all your prompts, and you may be relying on specific features of OpenAI. If you don't then you are at a disadvantage or at least slowing down your work.
If it does have considerable training data including prompt and response when people are interacting with itself then I suppose it isn't that surprising.
That does sound like self awareness, in the non magical sense. It is aware of its own behaviour because it has been trained on it.