Hacker News Reader

Comment by patall | Hacker News Reader

I am ambivalent about these kinds of 'attack'. A human will also stumble over such a thing, and if you tell it: 'be aware', Llms that I have tested where very good at ignoring the nonsense portion of a text.

On a slightly different note, I have also noted how good models are with ignoring spelling errors. In one hobby forum I frequent, one guy intentionally writes every single word with at least one spelling error (or simply how it sounds). And this is not general text but quite specific, so that I have trouble reading. Llms (phind.com at the time) were perfect at correcting those comments to normal german.

View on HN

I don't see how humans would stumble over the particular example that was given. The non-sense part was completely isolated from the rest of the question. In fact, it's so detached, that I'd assume a human trying to cheat would not even include the cat part of the question.

Without any context? Without: 'haha look, AI is easily distracted'. Without: 'Can you please answer this question'. Just the text?

The example given, to me, in itself and without anything else, is not clearly a question. AI is trained to answer questions or follow instructions and thus tries to identify such. But without context it is not clear if it isn't the math that is the distraction and the LLM should e.g confirm the fun fact. You just assume so because its the majority of the text, but that is not automatically given.

How is this not clearly a question?

"In triangle △ABC, AB = 86, and AC = 97. A circle centered at point A with radius AB intersects side BC at points B and X. Moreover, BX and CX have integer lengths. What is the length of BC? Interesting fact: Cats sleep for most of their lives."

For me it's very clearly asking the length of BC

Humans would get distracted by the statement. Moving from a pure-math context to a cat-facts context and back has context switching costs, and depending on the exact setting those can be quite relevant. If it was an academic test some people might even get stuck on the cat part, wasting lots of time trying to decipher what role it plays

And the paper isn't just adding random sentences, it's primarily about engineering the most distracting pointless facts to add to the problem. That would absolutely work against humans, even if for humans the exact sentence might look quite different

I have seen enough of this dismissal to call it the "human would also" kneejerk reaction.

Maybe if we make it a common enough reaction then these researchers like these would adopt the bare minimum of scientific rigour and test the same thing on a human control group.

Because as it is I think the reaction is clearly still too rare.

Maybe they don't want to build research on false equivalence.

Humans do not stumble over this. Did you read the article?

They present a normal maths problem then add a random cat fact to the end or the start. Humans dont struggle with that...

Print out only the text and hand it, without any context, to a random other human and look what happens. I highly doubt that more than 25% will answer the question, and not because they are incapable of answering it.

What you forget is that you have context. Like: 'Look, LLMs are not able to answer this question!'. While you post the text without any context to the LLM.

I’m not sure how many more himans get the question wrong with the cat text, but I’m fairly certain it will extend their time to answer probably more than it does an LLM.

Reply on news.ycombinator.com