That's why repeatable experiments are so important to science. Anybody can independently verify a testable claim.

Hahaha. Reliable confidence, isn't that a contradiction in terms? If a model makes mistakes but has great confidence estimation why wouldn't it make fewer mistakes in the first place, since it knows when it is wrong.

But if a LM looks up a topic and sees contradictory answers, and none of them is much more reputable, maybe it can still use that information to say it is inconclusive. Knowing a topic is controversial or not present in search engines is useful information. chatGPT would hallucinate, a search + chatGPT solution would refrain from hallucinations. It could also give references.

You shouldn't be so confident without knowing how these things work, there is a absolutely a simple and built-in way to model this... LMMs for example are simply calculating the next word or phrase sequence that is most likely given previous results and modeling information. So they can definitely tell you the combined liklihood that the answer is 'Peru is a cat' vs 'Peru is a country' and provide you the exact statistical likelihood of each.