Clever Hans (Intelligence Misattributon) | Hacker News Reader

Clever Hans (Intelligence Misattributon)

en.wikipedia.org

61 pointsanotherhue20 comments

I actually just posted an article that references Clever Hans (pretty sure it's also the name of a Grimm Brothers fairy tale, most of them have names like that).

It's actually not the best analogy for the current fad, I think "confused by a mirror" works much better. https://news.ycombinator.com/item?id=34850890 although I suppose the idea of attributing intelligence where there isn't any holds.

Clever Hans is a good analogy for (mostly deep learning) ML classifiers, the idea being a, say image classifier, can do extremely well despite not using any features that actually causally inform the decision being made. So when the background changes, or lighting, or some other thing, it fails utterly. This is actually true for almost all image classifiers.

I generally agree, Hans came to mind when I read a recent post where people discovered they had been nudging gpt towards answers without realising it.

https://news.ycombinator.com/item?id=35306095

Ah, interesting. I suppose that's a lot like what psychologists or psychics try and do, convince you to prompt them without you noticing.

Reminds me a bit of an old demo where you're trying to measure a certain signal shape in the presence of heavy noise. You get shown waveforms and decide if it looks like there might be a signal present or not, and the ones you approve get averaged to pull the signal out. Very quickly, even if all you are shown is zero mean gaussian noise, the average of what you OK'd starts to look like the waveform you're looking for.

"Der kluge Hans" is not directly a Brother Grimm title. There is a story "Der kluge Knecht", which translates into "The clever farmhand". The hero of the story is called in the story kluger Hans. There is another story called "Der gescheite Hans", which means exactly the same. Both stories are not fairy tales but droll stories about a simple-minded person.

The English translation includes "Clever Hans" which seems to be Der gescheite Hans.

https://www.gutenberg.org/cache/epub/2591/pg2591-images.html...

Eisenhans (Iron John)

Thank you for posting this, it’s validating to see that someone else sees this. What do you suppose the phenomenon is called when you get nervous that no one else sees it’s a mirror and you are concerned you will get stabbed with a pitch fork if you point it out?

I dont wanna take credit or anything, but this post appearing within 24 hours of me posting this comment https://news.ycombinator.com/item?id=35308954 makes me wonder if I'm partly reaponsible for this post now ^=^`

GPT-4 is Clever Hansing at web scale

I think of it more like extremely efficient large-scale Cargo Culting. It knows exactly what it should look like, without any conception of why or the originating factors that led to it.

I only knew of this story from looking up the name of this library on adversarial DL https://github.com/cleverhans-lab/cleverhans

Title is editorialized.

It is also inapplicable to GPTs, as they can answer questions, which the person asking does not know the answer to.

Case closed.

> as they can answer questions, which the person asking does not know the answer to

Search engines do this even better than ChatGPT does, and yet we don't call them "generally intelligent".

I don't see the relevance of this statement to the applicability of Clever Hans argument to GPTs.

emotional intelligence is intelligence, too. sheesh. the premise is flawed because the horse was engaged in a complex intelligent interaction with the person.

trickery aside, Hans was clearly intelligent by several metrics. mathematics are not the sole domain.

where do you see any suggestion that Hans lacked all intelligence?

"Intelligence Misattribution"

right in the title.

either it's clearly a gradient/spectrum, or this article isn't making the solid argument it might seem to be.

no, it's not a gradient/spectrum. The HN title is just missing extra words, like "Mathematical".

Nobody said that Hans had no intelligence, the point is that Hans doesn't have the kind of intelligence that was being attributed to him from the arithmetic demonstrations.

Intelligence misattribution does not follow from the story of the horse

More like accidental trickery

Sounds like Elon Musk fans.

Reply on news.ycombinator.com