Comment by bamboozled | Hacker News Reader

Comment by bamboozled | Hacker News Reader

Parent Full thread

But they're extremely relatable to us because it's regurgitating us.

I saw this talk with Geoffrey Hinton the other day and he said he was astonished at the capabilities of ChatGPT-4 because he asked it what the relationship between a compost heap and a nuclear bomb was, and he couldn't believe it answered, he really thought it was proof the thing could reason. Totally mind blown.

However I got it right away with zero effort.

Either I'm a super genius or this has been discussed before and made it's way into the training data.

Usual disclaimer: I don't think this invalidates the usefulness of AI or LLMs, just that we might be bamboozling ourselves into the idea that we've created an alien intelligence.

> Either I'm a super genius or this has been discussed before and made it's way into the training data.

If an LLM can tell you the relatonship between a compost heap and nuclear bomb, that doesn't mean that was in the training data.

It could be because a compost heap "generates heat", and a nuclear bomb also "generates heat" and due to that relationship they have something in common. The model will pick up on these similar patterns. They tokens are positioned closer to each other in the high dimensional vector space.

But for any given "what does x have in common with y", that doesn't necessarily mean someone has asked that before and it's in the training data. Is that reasoning? I don't know ... how does the brain do it?

> how does the brain do it?

It's a lot of organic matmuls. ;)

I mean that’s what sucks about Open AI isn’t it ? They won’t tell us what is in the training data so we don’t know. All I’m saying is that it wouldn’t be surprising if this was discussed previously somewhere in a pop science book.

That answer was close btw !

We used to have a test (Turing test) that could quite reliably differentiate between AI and our own species over the medium of text. As of now, we do not seem to have a simple & reliable test like that anymore.

Reply on news.ycombinator.com