> It’s completely unreliable because it sounds confident and plausible even when it’s wrong
This describes pretty accurately a non-trivial amount of people I have worked with.
This describes pretty accurately a non-trivial amount of people I have worked with.
In the end, why do people believe what they believe? The answer is it connects with what they already believe. That's it. If you had a diet of what we call science, you'll have a foothold in a whole bunch of arenas where you can feel yourself forward, going from one little truth to another. If you are a blank slate with a bit of quackery seeded onto it, you end up believing the stars predict your life and that you can communicate with dead people with a Ouija board.
CGPT doesn't have an "already believe". It just has a "humans on the panel give me reward" mechanism, and all it's doing is reflecting what it got rewarded for. Sometimes that's the scientific truth, sometimes it's crap. All the time it's confident, because that's what's rewarded.
Have most people invested time in making sure their web search results are accurate?
It is dangerous for sure, but that's what you got.
- there’s no cited facts
- it boldly proclaims its conclusion
- which you’d only be able to verify as an expert
…so I’m having trouble understanding what you’re complaining about with chatGPT, when that seems to be the standard for discourse.
The comment I was replying to was “the errors you’re talking about are probabilistic if you read the literature” my response is “no they aren’t I have read the literature.”
Note that I’m talking about a specific class of error and proving a negative is difficult enough that I’m not diving through papers to find citations for something n levels deep in a hacker news thread.
Toolformer: Language Models Can Teach Themselves to Use Tools: https://arxiv.org/abs/2302.04761
PAL: Program-aided Language Models: https://arxiv.org/abs/2211.10435
TALM: Tool Augmented Language Models: https://arxiv.org/abs/2205.12255
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models: https://arxiv.org/abs/2201.11903
Survey of Hallucination in Natural Language Generation: https://arxiv.org/abs/2202.03629
Survey of Hallucination in Natural Language Generation only provided promising methods for detecting hallucinations in summarization tasks, which are of course much easier to detect. Searching arXiv for a list of non-reviewed papers that sound like they might be related to the topic at hand is fun debate strategy. But no one else is reading this far into an old thread, so I'm not sure who you're trying to convince.
None of these paper prove your claims about hallucinations, and most aren't even trying to. However, even if the errors that I'm saying aren't meaningfully probabilistic aren't hallucinations.
This does not mean it is a problem that is impossible to fix.