Hacker News Reader

Comment by eru | Hacker News Reader

Humans also fail much of the time at 'multi jump' reasoning. You have to prod them.

Except no human (non-colorblind at least) past three years old thinks bananas have the same color as the sky (see the example given in the repo, that's a mistake literally no human could make)

Maybe not no human :) But probably 99.99% of them.

You're technically right, my two and a half had only be familiar with colors for six months, but I think it's fine to say that toddlers aren't reaching standard level of human intelligence ;)

For what it's worth, I tried it on ChatGPT and this was its response:

"The color of the daytime sky is commonly blue. The common household fruit that is also blue would be blueberries. Blueberries typically grow in acidic soil. The pH of the soil they grow in is usually between 4.5 and 5.5."

It can get this simple example right if it does chain of thought. If you ask it to just output the answer without answering other bullet points, it will very likely not get it right. Chain of thought is duct tape to actual reasoning, and the errors/hallucinations compound exponentially. Try to get chatgpt to reason about concurrent state issues in programming. If it's not a well worn issue that it already memorized, it'll be useless. It's also perfect, for instance at Advent of Codes that it's memorized, and near 0% accuracy on new Advents.

But as a human being I couldn't ever do it without chain of thoughts. First I would have to bruteforce come up with fruits that have the required colour. Otherwise I am just random guessing.

Also this problem I first tried to solve myself and I couldn't because I imagined sky to be light blue and blueberries are very dark if blue at all to me.

You could also go with grey or white etc for the sky.

Reply on news.ycombinator.com