> hallucinates and generates invalid data
The model is doing the exact same thing when it generates "correct" output as it does when it generates "incorrect" output.
"Hallucination" is a misleading term, cooked up by people who either don't understand what's going on or who want to make it sound like the fundamental problems (models aren't intelligent, can't reason, and attach zero meaning to their input or output) can be solved with enough duct tape.