Hacker News Reader

Hacker News

Comment by vbezhenar | Hacker News Reader

Parent Full thread

vbezhenar

GPT does not reason. GPT pretends to reason. It often is glaringly wrong. It's just a thing that plays with words.

Here's my conversation minute ago:

"Event C happened before events B and A. Event C happened between events A and B. What's the order for those events?"

The order of the events would be:

A → C → B

Event A occurred first, then event C occurred between events A and B, and finally event B occurred after event C.

"Event C happened before events B and A. Event A happened between events B and C. What's the order for those events?"

The order of the events would be:

C → B → A

Event C occurred first, then event B occurred after event C, and finally event A occurred between events B and C.

View on HN

I got the same answer and then I asked it

"Are you sure the original question made sense?"

Chat GPT: It's possible that the original question may have contained a logical inconsistency.

The statement "Event C happened before events B and A. Event C happened between events A and B" appears to be contradictory, as it is not possible for an event to both occur before and between two other events simultaneously.

If we assume that the statement was intended to mean "Event C happened after Event A and before Event B," then the order of events would be:

Event A Event C Event B In this case, Event C occurred between Events A and B, but after Event A. Therefore, the chronological order would be A, C, B.

"C happened before A and B" can also work in this if we assume it means the joint of A and B. Such that it is not the same statement that C happened before both A and B, individually.

Granted, language is ambiguous. But I don't think it gets points for assuming a mistake on input.

>> Granted, language is ambiguous. But I don't think it gets points for assuming a mistake on input.

So one person complains that it gave a "wrong" answer to contradictory input, and now you don't want it to guess at disambiguation. If we were this hard on kids when they're learning they'd be brought to tears.

BTW does anybody notice that this thing is handling abstract notions like "Event A, B and C" better than a lot of people do?

I made no real defense of the question, all told. It is written in a very hard to answer way. Just, if there is a way it could work, I'd expect that to be acknowledged.

I will again agree that logic is silly tough for language, though. Context changes the colloquial use of "and," and "or" all of the time.

"Event C happened before events B and A. Event C happened between events A and B. What's the order for those events?"

That doesnt make any sense. If its before B and A it can't also be between them

CACB

A is before C and B. A is between C and B.

That's the answer I expected.

May be it's some implementation nuances which forces ChatGPT to always assume the question having a correct answer. But second question does have a correct answer yet ChatGPT does not answer it.

From what I can tell, during the finetuning process OpenAI tried to make chatGPT try as hard as possible to follow instructions, and not point out when the question doesn't make any sense. I'm sure if they had used different criteria during RLHF and used different supervised examples it would have given that answer. It might make the performance worse on other tasks though.

Ah I see

I asked Bing Chat your question:

Me: "Event C happened before events B and A. Event C happened between events A and B. What's the order for those events?"

Bing Chat: "The order for those events is not possible because the statements are contradictory. If event C happened before events B and A, then it cannot happen between them. If event C happened between events A and B, then it cannot happen before them.

Can you please check the statements again and make sure they are consistent?"

> GPT does not reason. GPT pretends to reason. It often is glaringly wrong.

Being wrong is compatible with two hypotheses: 1. GPT does not reason and so correct answers are coincidence and happen no more than chance, 2. GPT reasons badly, which means it should still do better than chance.

We have ample evidence that GPT does better than chance, therefore GPT does reason, it just makes plenty of mistakes.

It reasons, just not always right. Give it time and these systems will become the oracles of humankind. A utility we need to use as an extension of our lives to succeed in the society we live in.

iiuc, GP’s point is that these models, in their current form, can’t guarantee confidence in their ability to reason about a scenario exhaustively.

“Give it time” and other dismissals are beside the point. The point seems to be that these models approximate results based on some hidden criteria, which means that although for a variety of practical purposes they fare better than people and are therefore preferable, there remain problems which are better suited to computing approaches that guarantee exhaustiveness… even though those other approaches sacrifice the malleability of neural networks.

Opponents of machine learning fear that ML will take over our thinking and it will stop us from properly thinking. But the same was said about writing things up by Socrates. Writing has transformed society and enabled wide spread of ideas. Maybe we now need AI to better parse those ideas? Sure, there may be errors during the process but I now think "AI reasonators" will actually make things better.

Incorrect results do not preclude reasoning. GPT is able to understand concepts, the relationship between concepts gives rise to reason and it is able to describe these relationships, therefore it reasons, even if the reasoning is not very good. Furthermore GPT is be able to supplement its reasoning with external information, which could take the form of 'reasoning engines'.

Reply on news.ycombinator.com