Please define what you mean by hallucination.
The OP's assertion is "make up stuff" is all LLMs really do and sometimes the stuff they made up matches reality - and sometimes it doesn't.
There are no separate modes of operation when GPT-4 is telling me about Javascript's "console.log" method, and when it's telling me about Rackets non-existing "sum" function. The network has no way of telling that the strands of probability are wearing thin under it's feet. The engine spins and churns out another token and we deem it "correct" or "incorrect".
Let's ask LLM to generate some text given a prompt. It replies with a bunch of words.
Let's ask LLM to generate some text given a different prompt. It replies with a bunch of other words.
In both instances, the model worked correctly, and as defined. It produced a bunch of words based on context, statistics and some forced randomness.
If we call one of them an hallucination because it does not match how we would expect a human to respond, we have to accept the other is also an hallucination because it was generated in exactly the same way, using exactly the same model.
If you disagree, ask the LLM to 'Compose a single-line avant-garde poem about rainbows, without mentioning rainbows'. Look at the reply. How would you tell whether it's hallucinating or not?
Also my objection wasn't that it hallucinates, my objection was to the OP's point that it only hallucinates. If I ask it what is 2*2 and it answers 4 would that be a hallucination or not?