I feel like some hallucinations aren't bad. Isn't that basically what a new idea is - a hallucination of what could be? The ability to come up with new things, even if they're sometimes wrong, can be useful and happen all the time with humans.
In humans, this is known as confabulation, and it happens due to various forms of brain damage, especially with damage to orbitofrontal cortex (part of prefrontal cortex). David Rumelhart, who was the main person who came up with backpropagation in a paper co-authored with Geoff Hinton, actually got Pick's disease which specifically results in damage to prefrontal cortex and people with that disease exhibit a lot of the same problems we have with today's LLMs: