Hacker News Reader

Comment by idiotsecant | Hacker News Reader

You shouldn't be so confident without knowing how these things work, there is a absolutely a simple and built-in way to model this... LMMs for example are simply calculating the next word or phrase sequence that is most likely given previous results and modeling information. So they can definitely tell you the combined liklihood that the answer is 'Peru is a cat' vs 'Peru is a country' and provide you the exact statistical likelihood of each.

View on HN

"So they can definitely tell you the combined liklihood that the answer is 'Peru is a cat' vs 'Peru is a country' and provide you the exact statistical likelihood of each..."

...in the context of the texts that the LLM is built on. Not in the context of the real world, where P('Peru is a country') = 1.0 and P('Peru is a cat') = #cats named Peru / #things in the world (or something).

Most of your world is text. Sure there's a sliver that isn't, but the reality you directly see is a tiny fraction of the reality you know from reports. Come to think of it, I've only ever seen reports of Peru, never actual Peru.

Actually I am confident exactly because I do know how LMMs work, and your comment fails to address the issue at all. Such models can't tell you anything useful about the probability that a particular statement is accurate.

That's how likely they are to occur near each other, not how likely either statement is true. Rude of you to preface your comment the way you did while making this error.

parent post isn't arguing which thing is capital T true (if such a judgement is even universally possible). They are talking about modeling statistical confidence, which is purely an emergent numerical property of data and makes no commentary on objective truth.

Reply on news.ycombinator.com