Hacker News Reader

Comment by stevage | Hacker News Reader

A lot like humans, really. People regularly cite something they read, or quote a stat that turns out to be just completely inaccurate. But if you look up the thing, then you have facts again.

View on HN

Sure, and by the same logic, if I was trying to cook a steak, I would not trust an arbitrary human to know the correct temperature off the top of their head; I'd want someone who I could trust had actual experience with the task I was trying to perform. The difference is that most humans are fully able to recognize whether they've cooked steak often enough to know the correct temperature off the top of their head, and they will say "I don't know" to most random arbitrary questions you ask them outside of their experience. I've yet to see an LLM product aimed at general usage for individuals be willing to say this without someone having to literally direct them to give that as an answer if they're not sure.

His point was that, just like humans, if you ask the LLM to look it up, it will give you the correct answer. Just like humans, if you don't ask them to look it up, you don't know what you're going to get.

And my point is that I've never met a human who confidently asserts incorrect information in such a broad range of domains rather than just admitting that they don't know

The point is that it's trivial to fix the problem of cooking the waterfowl or writing your code with existing tools. Just ask them to verify.

If you have to say "don't make up something" for every possible question you ask in order for it not to make up something, that's a massive usability issue for regular people. If saying "don't make up something" will still result in it making up something up some of the time, that also might be a massive usability issue depending on if "some of the time" means 0.0001% or 1%.

Having a natural language interface where you need to go out of your way to specify that you want an accurate answer rather than just a plausible one defeats the entire purpose of it being a natural language interface for normal people. In certain professional contexts, it can be useful, but I don't buy it at all that it makes sense to ask everyone in their everyday lives to go out of their way to specify that they actually want correct answers to their questions.

> If you have to say "don't make up something" for every possible question you ask

You don't. It goes in the system prompt.

In practice it doesn't though, for any actual products on the market today.

Put it in your own system prompt. They all provide tools for doing this. OpenaAI Custom GPTs, Gemini Gems, Claude Projects, etc. Or you can easily roll your own using their APIs.

Reply on news.ycombinator.com