Even if the answers accidentally happen to be correct, that's just the broken clock happens to be correct twice a day. The information value of answers by ChatGPT is zero.
Even if the answers accidentally happen to be correct, that's just the broken clock happens to be correct twice a day. The information value of answers by ChatGPT is zero.
As I said in https://simonwillison.net/2023/Apr/8/llms-break-the-internet...
> This is the thing I worry that people are sleeping on. People who think “these language models lie to you all the time” (which they do) and “they will produce buggy code with security holes”—every single complaint about these things is true, and yet, despite all of that, the productivity benefits you get if you lean into them and say OK, how do I work with something that’s completely unreliable, that invents things, that comes up with APIs that don’t exist… how do I use that to enhance my workflow anyway?
I encourage you to experiment more! Here are my notes so far: https://simonwillison.net/tags/llms/
> There was a boy called Eustace Clarence Scrubb, and he almost deserved it
> Everything starts somewhere, although many physicists disagree.
> It was a nice day. All the days had been nice. There had been rather more than seven of them so far, and rain hadn't been invented yet
> My father had a face that could stop a clock.
> It is important, when killing a nun, to ensure that you bring an army of sufficient size.
> In the myriadic year of our Lord—the ten thousandth year of the King Undying, the kindly Prince of Death!— Gideon Nav packed her sword, her shoes, and her dirty magazines, and she escaped from the House of the Ninth.
There's just no end to these.
And, of course, science is full of these too, one that jumps to mind is Shinichi Mochizuki's claimed proof of the ABC conjecture which has been proven flawed as it often happens but it was certainly a credible proof despite written in a language no mathematician have ever seen.
Weirdly, LLMs do that too.
Have you heard of the "Let’s be bear or bunny" pattern?
My iPhone hallucinated that the other day, running a LLM directly on the phone!
Yes but those are as every other thing they emit, indeed , "Let’s be bear or bunny", I heard it called AI hallucination I personally like to call it word salad.
> The man on the moon fell off his ladder one Tuesday, a common enough occurrence that nobody really paid it any mind. Truth is, gravity's always been a bit of a show-off, even in the star-dusted emptiness of space.
You can't rely in it being 100% correct, but that's very different from it having no informational value at all. When it comes down to it, you can't rely on anything being 100% correct. I recall finding multiple errors in textbooks in the past, and certainly Wikipedia is wrong about all kinds of things; that doesn't make them useless. It just means in situations where it's critical to be correct, you need to double-check. But often that's not necessary, and when it is, it's a lot easier to start with something and then verify it than to not have the tool in the first place.
Nonsense. The only way this would be true is if ChatGPT's answers were as likely to be correct as random answers. They are much, much more likely to be correct, however, so their information value is greater than zero by definition.
What you're claiming is equivalent to "search engines can find incorrect information, so search engines are worthless for information retrieval". Which is bollocks.
Which is the case
> They are much, much more likely to be correct,
They are not. It's just apophenia to find meaning in the word salad.
The irony of people complaining that LLMs confidently assert things that are completely wrong...