Hacker News Reader

Comment by ajross | Hacker News Reader

> Are we sure they know these things as opposed to being able to consistently guess correctly?

What is the practical difference you're imagining between "consistently correct guess" and "knowledge"?

LLMs aren't databases. We have databases. LLMs are probabilistic inference engines. All they do is guess, essentially. The discussion here is about how to get the guess to "check itself" with a firmer idea of "truth". And it turns out that's hard because it requires that the guessing engine know that something needs to be checked in the first place.

View on HN

> What is the practical difference you're imagining between "consistently correct guess" and "knowledge"?

Knowing it's correct. You've just instructed it not to guess remember? With practice people can get really good at guessing all sorts of things.

I think people have a serious misunderstanding about how these things work. They don't have their training set sitting around for reference. They are usually guessing. Most of the time with enough consistency that it seems like they "know'. Then when they get it wrong we call it "hallucinations". But instructing then not to guess means suddenly they can't answer much. There no guessing vs not with an LLM, it's all the same statistical process, the difference is just if it gives the right answer or not.

Maybe LLM's know so much that it makes it difficult to feel the absence. When someone asks me about the history of the Ethiopian region, I can at most recall very few pieces of information, and critically, there is an absence of feelings of familiarity. In memory research, familiarity signal is can prompt continued retrieval attempts, but importantly absence of that signal can suggest that further retrieval attempts would be fruitless, and that you know that you don't know. Maybe knowing so much means that there is near saturation for stop tokens...or that llms need to produce a familiarity like scoring of the key response of interest.

> Knowing it's correct.

I love the convergence with philosophy here.

This is the second reply that's naively just asserted a tautology. You can't define "knowledge" in terms of "knowing" in the sense of the English words; they're the same word! (You can, I guess, if you're willing to write a thesis introducing the specifics of your jargon.)

In point of fact LLMs "know" that they're right, because if they didn't "know" that they wouldn't have told you what they know. Which, we all agree, they do know, right? They give answers that are correct. Usually.

Except when they're wrong. But that's the thing: define "when they're wrong" in a way rigorous enough to permit an engineering solution. But you really can't, for the same reason that you can't prevent us yahoos on the internet from being wrong all the time too.

Simple, and even simpler from your own example.

Knowledge has an objective correctness. We know that there is a "right" and "wrong" answer and we know what a "right" answer is. "Consistently correct guesses", based on the name itself, is not reliable enough to actually be trusted. There's absolutely no guarantee that the next "consistently correct guess" is knowledge or a hallucination.

This is a circular semantic argument. You're saying knowledge is knowledge because it's correct, where guessing is guessing because it's a guess. But "is it correct?" is precisely the question you're asking the poor LLM to answer in the first place. It's not helpful to just demand a computation device work the way you want, you need to actually make it work.

Also, too, there are whole subfields of philosophy that make your statement here kinda laughably naive. Suffice it to say that, no, knowledge as rigorously understood does not have "an objective correctness".

I mean, it clearly does based on your comments showing a need for a correctness check to disambiguate between made up "hallucinations" and actual "knowledge" (together, a "consistently correct guess").

The fact that you are humanizing an LLM is honestly just plain weird. It does not have feelings. It doesn't care that it has to answer "is it correct?" and saying poor LLM is just trying to tug on heartstrings to make your point.

FWIW "asking the poor <system> to do <requirement>" is an extremely common idiom. It's used as a metaphor for an inappropriate or unachievable design requirement. Nothing to do with LLMs. I work on microcontrollers for a living.

> You're saying knowledge is knowledge because it's correct, where guessing is guessing because it's a guess.

Knowledge is knowledge because the knower knows it to be correct. I know I'm typing this into my phone, because it's right here in my hand. I'm guessing you typed your reply into some electronic device. I'm guessing this is true for all your comments. Am I 100% accurate? You'll have to answer that for me. I don't know it to be true, it's a highly informed guess.

Being wrong sometimes is not what makes a guess a guess. It's the different between pulling something from your memory banks, be they biological or mechanical, vs inferring it from some combination of your knowledge (what's in those memory banks), statistics, intuition, and whatever other fairy dust you sprinkle on.

So, if that were so, then an LLM possess no knowledge whatsoever, and cannot ever be trusted. Is that the line of thought you are drawing?

Reply on news.ycombinator.com