GPT-4 Understands

This type of argumentation just shows disregard to science. You can actually figure out what GPT-4 understands, but you don't do it with single examples.

Take a concept. Make a test battery of questions that probe the concept using different words, different approaches, situations etc. Just like psychologist would do with 4 year old.

You quickly realize that GPT-4 has not clear conceptual understanding that would show as consistency.

> The implications are vast. We may be able to translate between languages that have never had a “Rosetta Stone”. Any animals that have a true language could have it decoded. And while an LLM that’s gotten an 8 year old’s understanding of balancing assorted items isn’t that useful, an LLM that’s got a baby whale’s grasp on whale language would be revolutionary.

Given that almost no-one trusts these LLMs for anything serious beyond creative / BS generating uses, this article does a lot of glossing over the reliability of their outputs and focuses more on such utopian promises with claims of 'understanding' which is in this article is beyond absurd and nonsensical.

The reality of LLMs is that they cannot reason. [0] It is known that LLMs perform incredibly worse in mathematical reasoning and understanding which it consistently hallucinates the wrong answers [1] and even when you tell it to correct itself, it pretends to identify its mistake and regurgitates another incorrect answer and its answers get even worse further down the conversation. [2]

Of course to the typical user, it's easy to be fooled by a stochastic parrot pretending to demonstrate reasoning. But experts understand that these systems cannot transparently explain themselves and are AI sophists which fundamentally cannot be trusted.

[0] https://arxiv.org/pdf/2302.08399.pdf

[1] https://arxiv.org/pdf/2301.13867.pdf

[2] https://arxiv.org/pdf/2310.01798.pdf

"understands" is not, as the author suggests, a matter of "semantics", it's one of the key concepts here.

Objectively, don't we know that GPT-4 was built and trained in a very similar way to GPT-3 and 3.5? Genuine question--that's true right?

If so, the difference is one of degree, as opposed to kind.