This is an underrated observation. It's probably a mathematically similar phenomenon happening in GPT. And/or it discovered meth.
I just hope Vince Gilligan will direct Breaking RAG.
Not far from MethGPT!
The extent to which it will be accurate depends on how much of sample transcripts were in its training data, I suppose.