> right down to the typo "you name" rather than "your name", which seems unlikely to happen if it were making it up out of thin air.
Why is it unlikely? Why does prompting it different ways and getting the same result make it unlikely?
Why is it unlikely? Why does prompting it different ways and getting the same result make it unlikely?
A hallucination, when it comes to LLMs, just means "the algorithm picking most likely next tokens put together a string of tokens that contains false information". It doesn't mean the LLM is having a novel false idea each time. If the first time it hallucinates it thinks that that misspelling is the best next-token to use, why wouldn't it keep thinking that time and time again (if randomness settings are low)?
Obviously they're a black box so it's possible there could be some very rare edge cases where it happens anyway, but it'd be a complete fluke. Changing the prompt even superficially would essentially cause a butterfly effect in the model that would prevent it from going down the exact same path and making the same mistake again.