> OpenAI arguably already has AGI.

Just no.

> GPT-3/ChatGPT is much better than the average human at coding, text summarization, etc. (“system 2” cognitive tasks).

I don't think you even believe what you have just said there.

Given it hallucinates 98% of the time and produces incorrect code and also is unable to reason about why it is wrong, it is completely untrustworthy and requires humans to triple check the output. It cannot be used unsupervised and needs constant review.

We haven't even gotten started with the nonsense that it outputs or how it cannot transparently reason about its own decisions in the most serious of applications such as finance, law and medical uses. Meaning that its use is instead reduced to just sophistry generation.