[flagged]
Consider: GP would've been much more correct if they said "It's just a person on a chip." Still wrong, but much less, in qualitative fashion, than they are now.
It's a person in the same sense as a Markov chain is one, or the bot in the reception on Starship Titanic, i.e. not at all.
FWIW, I prefer my "little people on a chip" because this is a deliberate riff on SoC, aka. System on a Chip, aka. an actual component you put when designing computer systems. The implication being, when you design information processing systems, the box with "LLM" on it should go where you'd consider putting a box with "Person" on it, not where you'd put "Database" or any other software/hardware box.
Edit: unless we are talking about MongoDB. It will only keep your data if you are lucky and might lose it. :)
It's not just the weirdness in Mongo that could exhibit non-deterministic behaviour, some common indexing techniques do not guarantee order and/or exhaustiveness.
Let it go, LLM:s and related compression techniques aren't very special, and neither are chatbots or copy-paste-oriented software development. Optimising them for speed or manipulation does not change this, at least not from a technical perspective.
It's like a JPEG. Except instead of lossy compression on images that give you a pixel soup that only vaguely resembles the original if you're resource bound (and even modern SOTA models are when it comes to LLMs), instead you get stuff that looks more or less correct but just isn't.
An LLM chatbot is not like querying a database. Postgres doesn't have a human-like interface. Querying SQL is highly technical, when you get nonsensical results out of it (which is most often than not) you immediately suspect the JOIN you wrote or whatever. There's no "confident vibe" in results spat out by the DB engine.
Interacting with a chat bot is highly non-technical. The chat bot seems to many people like a highly competent person-like robot that knows everything, and it knows it with a high degree of confidence too.
So it makes sense to talk about "hallucinations", even though it's a flawed analogy.
I think the mistake people make when interacting with LLMs is similar to what they do when they read/watch the news: "well, they said so on the news, so it must be true."
It's precisely like a database. You might think the query interface is special, but that's all it is and if you let it fool you, fine, go ahead, keep it public that you do.