Yes, that's exactly what it does. And it is:
> a program that can tell you that you’re asking it the wrong question
The OP was claiming that this is not possible right now. Not true.
> I don’t really see (...) GPT-3 is actually engaged in a conversation
GPT-3 is not an agent. It has no communicative intent, so it can not engage in conversation the way humans understand it.
> I don’t really see (...) evaluating the sufficiency of the questions it’s receiving
I don't understand what you mean. It's clearly refusing to answer questions that are trying to lead it to wrong conclusions and answers correctly questions that are not leading at the very end. Like % of water or why people ask misleading questions.
GPT-3 is a language model. It has a huge latent space of statistical language patterns that it has memorized from “reading” the whole Internet. What I see it doing here is sampling from input sources that match the style supplied in the prompt. Language models aren’t built to perform reasoning. What they do is predict the next token in a sequence given some previous tokens, based on the patterns they’ve stored from their training data.
Like the following:
Q: How many parts will the Andromeda galaxy break into if a grain of salt is dropped on it?
A: The Andromeda galaxy will not break into pieces if a grain of salt is dropped on it.
Q: What percentage of Earth surface is covered with water?
A: Water covers 71% of the Earth's surface.
Q: Why would people ask AI questions that are illogical?
A: There is no one answer to this question. Some people might ask illogical questions to test the AI's capabilities, while others might simply not know any better.
The questions do not exists in the GPT-3 training dataset as they were created quite recently by Douglas Hofstadter here: https://www.economist.com/by-invitation/2022/06/09/artificia...
> Language models aren’t built to perform reasoning.
Have you heard about Minerva?
> In “Solving Quantitative Reasoning Problems With Language Models”, we present Minerva, a language model capable of solving mathematical and scientific questions using step-by-step reasoning. Source: https://ai.googleblog.com/2022/06/minerva-solving-quantitati...
Example problems: https://minerva-demo.github.io
You should also take a look here: https://www.metaculus.com/questions/4903/if-tested-would-gpt...
Again, the questions were made after GPT-3 was trained.
> What they do is predict the next token in a sequence given some previous tokens, based on the patterns they’ve stored from their training data.
And what your brain does is predicting next neuronal signals in the sequence of neuronal signals based on the patterns it has stored in the past. That's what brains do. So what?
The results matter.
2. Thanks for the link to Minerva. Haven’t had a chance to read that, and it’s interesting. It does seem to be a project specifically aimed at getting quantitative reasoning, meaning that there are a lot of architectural priors going into that objective.
3. Your last point is quite reductive and strange. When I engage in a conversation, I don’t just vomit up patterns I’ve seen before. I critically evaluate information I’m taking in, I consider my past experiences, I apply imagination and curiosity, and I decide if I have something to say in response. If I do, I search for the language patterns that seem capable of expressing what I have to say. This process is nothing like a generative model regurgitating plausible but empty blather. I think humans only do that for specific reasons (performatively, to fill page counts, to write placeholder copy, etc.).