Hallucinations are a result of using statistical models on data. Not just transformers have this issue.