Hacker News Reader

Comment by _nalply | Hacker News Reader

This hallucination is sobering and thought-provoking.

ChatGPT creates a bubble around your reasoning. If you discover hallucination it tries to correct itself, but most often it just creates different nonsense.

If you are not very well-versed and believe you understand it you could fall for it.

If you program you at least can try out and give feedback till it finds a solution that works. However this is not better than those programmers who introduce a lot of security issues.

ChatGPT is at best for finding ideas which are often bogus but could inspire you to useful ideas (a form of brainstorming) and at worst just useless or even dangerous.

View on HN

Or maybe just use GPT-4 that doesn't make up this kind of garbage hallucinations.

I don't understand how people on HN keep conflating the two.

I don't use GPT-4, but as far as I know GPT-4 also hallucinates but not as much as ChatGPT.

Real GPT-4 (that means excluding whatever Bing and Phind claims is GPT-4) doesn't hallucinate especially when it comes to STEM questions like OP.

Yes it does.

Edit: Indeed, since you trust GPT-4 so much here is what it has to say on the subject:

> Does GPT-4 hallucinate?

> No, GPT-4 doesn't "hallucinate" in the way that humans or animals might experience hallucinations. However, the term "hallucinate" is sometimes used informally to describe when GPT-4 (or similar models) produces outputs that are not grounded in its training data or are factually incorrect. This typically means that the model is generating content based on patterns it has recognized but that might not be accurate or contextually appropriate.

> Does GPT-4 hallucinate in the latter sense?

> Yes, in the latter sense, GPT-4 can "hallucinate." It can produce outputs that are not grounded in its training data or are factually incorrect. This occurs because GPT-4 is predicting the next word based on patterns in the data it has been trained on, and sometimes those predictions might not be accurate or contextually appropriate.

Hah! A variant of the liar's paradox!...

For those who don't know about this paradox, let me explain it. We have now the situation that GPT-4 itself states it can hallucinate. Let's say this is true. Then case closed. GPT-4 can hallucinate. But what if this is not true? Then case closed. We have proven that GPT-4 has hallucinated in at least one case.

It does for sure

Reply on news.ycombinator.com