Hacker News Reader

Comment by qup | Hacker News Reader

It's so weird that people use questions that are well-known for duping humans, who we all consider to be general intelligence.

Getting this question wrong doesn't say much about the intelligence of humans, why would it say something about the AI?

View on HN

We use variations on questions that are well known for duping inattentive humans, to test a system that we expect a priori to be incapable of such inattention.

Unless "getting easy things wrong sometimes" is an inherent property of intelligence, we should expect that a properly "intelligent" computerized system would never err on problems far below its level of comprehension - unless we had some reason to believe it "wanted to", and as of yet I see no reason to believe this is even possible in principle.

Humans err, broadly speaking, for two reasons: genuinely reaching the limits of their comprehension, or trusting "system 1" (in Kahneman's analysis) too much.

Because for things like the Putnam questions, we are trying to get the performance of a smart human. Are LLMs just stochastic parrots or are they capable of drawing new, meaningful inferences? We keep getting more and more evidence of the latter, but things like this throw that into question.

It is perfectly possible for the first AGI to be stupid. A moron. In fact, I'd bet that's fairly likely.

I would agree if we weren't starting with LLMs for a baseline. The first AGI will know at least as much as LLMs, IMO, and that's already not-stupid. Especially once they can separate out the truth in their training.

Okay, but you just invented your own bar of "smart human" to be the universal bar (I don't share that opinion).

Also, lots of smart humans can't do the freaking Putnam, it doesn't make them stupid. It makes them non-experts.

Reply on news.ycombinator.com