Not sure if there is a great deal of maths to understand. The output of an LLM is stochastic by nature, and will read syntactical perfect, AKA a hallucination.
No real way to mathematically prove this, considering there is also no way to know if the training data also had this “hallucination” inside of it.