If current scaling trends hold, it may not be possible to distinguish a hypothetical GPT-6 vs. a human over reasonably sized conversations/text production tasks. Granted, such a GPT-6 model may be larger than any commercial application could support, and possibly beyond the range of research oriented funding as of today.
GPT can never be an intelligent agent because it’s not an agent. Also, the increased training time to produce a really big model makes it less like an intelligence, which has low marginal cost to learn something new.
The reason GPT looks intelligent is because you’re the one providing that; it’s just a big blob of inanimate knowledge.
1. During training time, which is very expensive in terms of input text and money - plus, model training often completely fails which is why you have to do checkpoints, hyper parameter searches, etc.
2. During inference, but it doesn't have arbitrary thinking and memory abilities like a human does, it only has however much space is in the input tokens + its weights. There are thoughts that can't fit even in an optimal model.
GPT isn't just a model, it also has that sampling system which is a regular computer program (as opposed to a learned one), which does give it extra abilities.
Are you saying that GPT-6 would be less intelligent than a human, because we could give it an arbitrary deadline within which to answer the question, while not restricting a human that way? That doesn't sound like a fair test.
I'm probably misrepresenting your position here, and maybe what you're trying to say is "If you gave both GPT-6 and a human as long as they needed go away and think about a problem, the human could use that time to sit and think creatively about the problem, whereas GPT-6 would either know the answer, or not".
That's a reasonable intuition and a fair experiment, but I think it hinges on the question of "What does a human do when they go away and think about a problem?". Could it be that they are just clearing their mind of other distractions, and trying to look at the problem from multiple perspectives, which is something that a machine could do more efficiently?
In other words, the fact that a human might need to go away and take arbitrary time to think about a problem is a sign that it is the human that lacks efficient general intelligence, not this GPT-6.
> can go out and perform experiments,
Again, it seems unfair to compare GPT-6 in a deliberately sealed box to a human who is allowed to go away and talk to other humans and use tools and resources that we wouldn't trust GPT-6 with.
Putting it another way, if you met someone who had never performed an experiment before, would you think they were not intelligent?
> can Google/GPT things themselves, etc.
This is an almost Kafkaesque criticism. Are you saying GPT-6 isn't intelligent because it doesn't have access to GPT-6's intelligence, but humans do? Or the fact that humans can go away and use Google means they are more intelligent than GPT-6 which has Google search results built into its "brain" in the first place?
> it’s just a big blob of inanimate knowledge.
You're just a big blob of inanimate knowledge. Ahem, sorry, what I mean is: Why does knowledge have to be "animate" to be legitimate?
Are there any efforts to make one gpt talk to another ? Can the older versions eg. gpt1 and gpt2, talk to each other, and eventually become more performant than gpt3 ?
On the other hand, the lottery ticket hypothesis says that every good big network contains a good or better smaller network you could extract, if only you knew where it was. (So the reason big models are good is there's more opportunities for the smaller models to appear.)
The basic design of an ML model is just one really big matrix multiplication that always takes the same time - I'm saying we haven't given GPT the ability to not do that, so it always has that arbitrary default.
It does have a system where it does beam search through the model instead of simply running it once, but it's not fully recurrent.
> That's a reasonable intuition and a fair experiment, but I think it hinges on the question of "What does a human do when they go away and think about a problem?". Could it be that they are just clearing their mind of other distractions, and trying to look at the problem from multiple perspectives, which is something that a machine could do more efficiently?
Well, some problems just take longer to think about or need knowledge that isn't already written down.
> Putting it another way, if you met someone who had never performed an experiment before, would you think they were not intelligent?
I think a prisoner or someone like that could demonstrate they're intelligent. If you're in the room with them, I mean, not like the Chinese room problem. But I think pure "intelligence" is just going to be less interesting that it looks, because it can't accomplish many tasks you'd expect it to if it can't actively interact with anything.
> Are you saying GPT-6 isn't intelligent because it doesn't have access to GPT-6's intelligence, but humans do? Or the fact that humans can go away and use Google means they are more intelligent than GPT-6 which has Google search results built into its "brain" in the first place?
Human + GPT will be more productive than GPT (at writing things that are actually logically correct) is all I mean. This isn't true of all AI - humans + chess programs are slower than chess programs - it's just because of the specific structure of GPT.
> You're just a big blob of inanimate knowledge. Ahem, sorry, what I mean is: Why does knowledge have to be "animate" to be legitimate?
I mean, it's cool that it's a big blob of knowledge, but that isn't the human capacity of "intelligence", it's "memory".
When it combines different inputs from its memory into a single one fluently, that shows something like intelligence, but it was very expensive to make that happen during training time. And GPT has flaws there you can't fix; for instance, it thinks orioles (the bird) and the Baltimore Orioles are the same thing:
She uses a GPT-2 detector there, and you definitely do not know that that is a 'flaw that you can't fix' in all possible GPT-like models.
Btw, GPT-3 has the same issue, just tried it. (This is cherry picked; I used the Q&A example, removed the line about replying Unknown to trick questions, and tried ~5 prompts.)
-
I am a highly intelligent question answering bot. If you ask me a question that is rooted in truth, I will give you the answer.
Q: What species are the Baltimore Orioles? A: The Baltimore Orioles are a species of bird.
Did you need to remove this because it identified your question as a trick question? If so, doesn’t that indicate that the model does in fact know the difference?
Here's another try. (Notice it didn't answer the first question.)
--
I am a highly intelligent question answering bot. If you ask me a question that is rooted in truth, I will give you the answer. If you ask me a question that is nonsense, trickery, or has no clear answer, I will say it's a guess, but still try to answer.
Q: What species are the Baltimore Orioles' players? A: The Baltimore Orioles are a professional baseball team in the American League East Division. As of the 2019 season, the Orioles have 25 players on their roster. These include 12 pitchers, 2 catchers, 6 infielders, and 5 outfielders. The Orioles have had players from all 30 Major League Baseball teams, as well as players from countries such as Canada, Cuba, the Dominican Republic, Japan, Mexico, Panama, Puerto Rico, South Korea, Taiwan, and Venezuela.
Q: What do they like to eat? A: I'm not sure, but I guess they like to eat insects.
For all the handwaving people like to do, this is the bottom line, and it's not clear to me why researchers haven't spent more time clearly formulating this, vs a lot of proving that various tasks and games don't actually require intelligence.
As best I can formulate it, "AI" can't "escape" its programming, and just a neural network even less so. There is no mental model of the world to rationalize against, and if there was, it would be pre-defined or coded in some way that it still couldn't be escaped. Whatever tasks a ML model has been deliberately programmed with just have nothing in common with intelligence. Scaling doesnt change that.
I subscribe to the idea that with ML we may have understood how some of the sensory mechanisms work, like visual recognition, but we're not actually closer to the intelligence part.
Not that I agree with the author but 'AIs are sandboxed and resourced constrained therefore safe' doesn't feel very convincing to me after reading that story.
I don't think you can back that claim up. ML models as often used can't escape their programming. But I don't see any fundamental reason you can't loop one back on itself.
Isn't this already done to some extent by using a model to assess its own output versus the current environmental state? Recurrent models combined with reinforcement learning actually look a lot like what you're claiming can't be done.
> There is no mental model of the world to rationalize against
There's a theory and some associated research about a model that performs well necessitating an internal model that accurately reflects the environment in which it operates. Or something like that. Sorry I can't recall the paper off the top of my head.
> Whatever tasks a ML model has been deliberately programmed with
Yeah, no, that's not the case. See the entire field of reinforcement learning.
Yes, you can do this. And I believe you could recreate human intelligence if you made something exactly like a human and put it in an environment exactly like a human - but for some reason the AGI theorists think AGI via AI research is scary and "N"GI via having children isn't.
But GPT doesn't have those features, which is why it's not one of them.
> Yeah, no, that's not the case. See the entire field of reinforcement learning.
ML models (especially big ones) fluidly combine learning with memorizing, which is cool, but it's more like "compression" than "programming".
It's already almost there.
GPT-3 has the ability to generate short or long. If you ask it "let's think step by step" it generates a chain of thought, dividing the task into steps and doing them one by one, almost like an agent.
On the other hand, if you strap GPT-3 on an agent in a 3D Home Environment it gains the ability to perform multi-step tasks zero shot. So it's quite ready for it.
If you train a reward model based on supervised data, you can finetune GPT-3 with it. This is how Instruct GPT-3 was created, through text-based RL.
I'd say GPT-3 is very close to being an agent, it just needs to be put in a body in an environment and trained a bit longer.
> On the other hand, if you strap GPT-3 on an agent in a 3D Home Environment
GPT-3, just as any probability distribution, can inform an agent's actions but that doesn't make it one. WebGPT3 is an agent though.
WebGPT does need safeguards because eg it could start querying websites and end up hitting their /delete APIs. But that's not really an "AI alignment" problem; same thing happens with GoogleBot.
A GPT model already has many ways of thinking and self-distilling; for example, it can give itself a lot more time to think using 'inner monologue' techniques, which make it capable of calculating out step by step, using external tools like Python etc: https://www.gwern.net/docs/ai/gpt/inner-monologue/index A GPT model can also already Google things, see WebGPT https://openai.com/blog/webgpt/ (itself merely one entry in a rapidly-expanding area of retrieval models https://www.gwern.net/docs/ai/retrieval/index ). Enabling them to use web tools is likewise an active and increasingly successful area of research https://arxiv.org/abs/2202.08137#deepmind , with startups forming for that purpose: https://www.adept.ai/post/introducing-adept It can't 'run experiments', but the setting of reinforcement learning would allow that, and Decision Transformer (literally just a GPT trained on RL data and then given a prompt) is one of the more exciting directions - you may have seen Gato https://www.deepmind.com/publications/a-generalist-agent but have you also seen Multi-Game Transformer https://sites.google.com/view/multi-game-transformers ? Oh, that's just a single agent and you're not worried? Well that's fine because DT is so general it's almost trivial to make it do multi-agent RL as well https://arxiv.org/abs/2112.02845 https://arxiv.org/abs/2205.14953
Things are moving fast, and few people know all of the things that a GPT can do. (By the way, did you know that this 'big blob of inanimate knowledge' also knows what it doesn't know and can give well-calibrated predictions of accuracy https://arxiv.org/abs/2205.14334 ?)