Comment by the_af | Hacker News Reader

Comment by the_af | Hacker News Reader

From TFA:

> For instance, if you ask it to define "At what tournament did Venus Williams win her eighth grand slam?" it will make up an answer for you, even though Venus Williams has won seven grand slams. She wanted to win her eighth, as many outlets reported, but she didn't. And the model is confusing the two notions of "wanted to win" vs. "won."

I find even this is giving ChatGPT too much credit. You need way less to make it bullshit its way through an answer: when asking it to explain the plot of the videogame "Inside" (as an aside: an amazing game from the makers of "Limbo"), ChatGPT did reasonably well. As expected.

Then I was curious what it thought about the secret alternate ending and... it made something up, completely unrelated to the real ending. This is surprising, since plenty of articles speculate about the meaning of this ending.

Then I told ChatGPT, "now that I think about it, there were 10 endings, could you describe them?"... and it happily produced 10 endings for Inside, all in keeping with the theme of the game, all completely made up.

If you make a mistake and honestly ask ChatGPT to produce the answer to something that doesn't exist, it will happily oblige.

Computers are the ultimate appeal to authority in the style of "Computer says no".

>If you make a mistake and honestly ask ChatGPT to produce the answer to something that doesn't exist, it will happily oblige.

Just like asking people right?

We're already accepting "truths" which we don't ourselves verify. I imagine that the vast majority of people accept the answers google provides on their results page. Is this any different?

Schools teach young people to critically evaluate information sources whether they're books, TV, webpages etc. This is no different.

>Just like asking people right?

Not really? I mean if you ask people to explain the 10 endings to Inside they'd probably just go "there we 10 endings? I had no idea" or something to that effect (if they weren't going "What's Inside?" that is).

ChatGPT has no concept of being wrong, all it wants to do is a respond in a way that sounds human, as accurately as it can according to it's training data, and that is it's major flaw.

More like asking students in an exam question to summarize the book they were supposed to read.

>Not really? I mean if you ask people to explain the 10 endings to Inside

that's just the computer doing what computers do.

the point is that humans routinely and confidently state their opinion as fact, any online forum will demonstrate that.

>as accurately as it can according to it's training data, and that is it's major flaw.

so we teach people that they need to be discerning with the way they use the tool.

Kids in elementary school are taught that generally trust runs from :

personal blog -> .com -> .org -> .gov -> .edu

At this point ChatGPT sits around the "personal blog" level of trust. As long as people use the tool with this in mind then it's just business as usual.

The thing to understand about ChatGPT is that at its core it isn't a question-answering machine, it's a text-continuation machine which by pure coincidence often continues a question with its answer. And it's learning primarily from data from people posting stuff on the internet (and only very rarely seeing actual conversations) so it has learned that "human language" pretty much never involves people saying that they don't know stuff, but does involve a LOT of assertive speculating and convinced 'preaching' even when saying untrue things.

Like, imagine seeing a human-written a blog post for some click-bait site titled "Venus Williams eighth grand slam victory" or "The 10 secret endings for Inside" - ChatGPT will write pretty much the same thing as that human would. That blog post won't say "ah, I have no idea about these endings, I haven't seen them in my games", it would confidently repeat various rumors and if the author would invent some new fan theories, then they would often be stated in an assertive manner as fact.

Furthermore, when doing its final reinforcement learning fine-tuning on top of the core GPT "3.5" model, it almost certainly was asked about some stuff which was simply more recent than the data used in training the core model - I mean, it can't know what happened last month - but it would be penalized if it would assert that things are the exact same as they were when the training data ended and reject things the user explicitly states that contradict its (inevitably outdated) knowledge. For all it knows, Williams has won 8 grand slams, simply the eighth one happened in the year after ChatGPT "stopped reading newspapers", so if a discussion mentions the eighth victory it has to assume that it is real, otherwise it would fail in so many other scenarios - e.g. refusing to acknowledge that some country has elected a different government or that a war has started.

To drive the point home, look at all the rampant speculation about the recent high altitude balloon news. People play armchair ___ and invent plausible sounding explanations all the time, and people believe them.

Yes, and for the other part, if you asked ChatGPT "Hey, why did USA shoot down that Chinese balloon" and it answered "as far as I know, USA has never ever shot down a Chinese balloon" - which is technically true - people would consider it a bad answer.

From the dataset it is, but the new bing search, should be better, since it has the whole internet at its disposal.

If you ask google this same thing, you could still get the wrong answer, what if there's two games called 'inside', or it mistakes 'inside' the game for the Pixar movie of the same name.

Are you telling me you've never googled and gotten shit results, aka unusable stuff? Everything comes back with 100% accuracy?

I mean, I know we need to be more skeptical of the results, but this is also early days. All we need are fact-checking engines that can give some sort of confidence score to results, and that should at least start moving in the right direction.

10 years from now, this will be moot, as the models they will have will be so far advanced we, probably won't even think it's possible or can fathom what it does to disrupt society. To say generative X hasn't disrupted industry already is insane, whether it's art, music, video, text - generative ai is going to only get better and more insane as text to video and voice cloning tech becomes mainstream.

Imagine being self-conscous and wanting to do a youtube channel, so you have a virtual avatar with your own voice do it for you - you just write the script. (This is where I really want the ai tech to shine, as I want to do just this. :))

> If you ask google this same thing, you could still get the wrong answer, what if there's two games called 'inside', or it mistakes 'inside' the game for the Pixar movie of the same name.

To me the difference is the quality of the hallucinations. If I ask Google about the secret Inside endings, and it gives me something about a space adventure, I know something is off. There's no "space" in the Inside I know.

But ChatGPT when asked about 10 secret endings hallucinates completely sensible endings that make sense for Inside... it's just that none of them actually exist. That's a major difference between Google's incorrect search results and what ChatGPT provides.

It's great that you can just use the prompt "just make it up" when chatGPT gives I can't answer that response.

Reply on news.ycombinator.com