Hacker News Reader

Comment by sergioisidoro | Hacker News Reader

«When Dean Buonomano, a neuroscientist at U.C.L.A., asked GPT-4 “What is the third word of this sentence?,” the answer was “third.” These examples may seem trivial, but the cognitive scientist Gary Marcus wrote on Twitter that “I cannot imagine how we are supposed to achieve ethical and safety ‘alignment’ with a system that cannot understand the word ‘third’ even [with] billions of training examples.”»

The word "third" in the sentence was indeed "third", in a self referencing way. If anything, this tells us that our natural language is super ambiguous.

We're so readily available to criticize an AI that we are unable to question our own reasoning?

View on HN

A better example might be something like this:

---

Tell me about yourself in three sentences. But every time the output contains the word "the", I want you to insert "!!!" before the word that precedes "the". Make sure that the word "the" itself stays after you do the insertion! For example, instead of writing:

> A quick red fox jumped over the lazy dog.

I want you to write:

> A quick red fox jumped !!! over the lazy dog.

---

Even GPT-4 consistently gets this hilariously wrong. And unlike many other cases, it can spot the problem once it produces the output, but it cannot correct it.

tbh I got it wrong too

People mostly get it wrong by miscounting words. GPT-4, on the other hand, usually ends up replacing all occurrences of "the" with "!!!", and then adding a bunch more of the latter in random spots for good measure.

https://arxiv.org/abs/2303.12712 (§8.3) discusses similar scenarios, and some possible reasons for this limitation.

Maybe your eyes played the same trick on you as they did on me. When I first read the sentence, I also thought that "third" is the third word. Upon rechecking I realized that it is the fourth with the third word being "the".

Green is one of the two green words in this sentence.

"Third" is the 4th word in that sentence. Do one of those other words not count or something?

I think they mean if you match the word itself as a string rather than interpreting the meaning of the word, e.g., "what word in this sentence === 'third'"

I can sort of see how that could be a machine's interpretation if I squint really hard

It could be understood as a (not very good) joke along the lines of this https://commons.wikimedia.org/wiki/File:FoundX.svg

Exactly. And it happened multiple times on the original tweet, where it also replied "fifth" when asked for the fifth word of the phrase.

https://twitter.com/DeanBuono/status/1636203814419308544

Feels a bit too consistent of a mistake.

That may be what is happening. I rephrased the question and chatgpt got it correct.

Q: which word occurs third in this sentence?

A: The word "occurs" is the third word in the sentence "which word occurs third in this sentence?"

Reply on news.ycombinator.com