Why didn't DeepMind build GPT3? | Hacker News Reader

Why didn't DeepMind build GPT3?

rootnodes.substack.com

26 pointssebg20 comments

Very interesting read. The author’s point about traditional academic conferences not accepting papers for GPT3 seems plausible given that Meta AI Chief Scientist Yann LeCun tweeted about why ChatGPT wasn’t impressive

A less well-known problem is by no means a trivial one. Only time can tell whether prediction of protein folding or prediction of next words for prompt is more important. But they are like apple and pear.

May be you should ask GPT whether word or protein is more important :)

Proteins were made so words could happen

The next challenge is not how to make a smarter ChatGPT. It's how to contain it (and its numerous clones).

Because if you think someone is not already at work to use a GPT bot to create the next Stuxnet, you are fooling yourself.

I think the cat is out of the bag unfortunately.

I think the next challenge is to close the loop on ChatGPT and similar programs. So not only do they suggest code, then they run that code, see if it does what they expect and then fix that code.

Suggesting code that doesn't have to work is like 1% of the problem, the other 99% is validating and fixing it.

We're nowhere near there yet.

I wonder how far we are.

I do get the code from ChatGPT, I run it, I give feedback to ChatGPT, I then fix the bugs GPT points out. This loop mostly works. Although it often needs external information in order to resolve some of the ambiguities.

Also ChatGPT apologizes when it is wrong, although it isn't always able to fix the issue.

I feel that some structure around this will cause ChatGPT + Github Copilot to jump up another order of magnitude in productivity/effectiveness. Maybe two orders of magnitude.

What do you think were the major challenges involved in creating Stuxnet and why do you think a GPT bot will make the next one significantly easier?

ChatGPT is already able to write programs in various languages. I guess it was trained on open source programs like those in GitHub. Someone (let's say the NSA) can train it on a large database of viruses, warms, known exploits, etc, and ask it to come up with a new virus with some given parameters. The NSA is either doing that or not. Russia's hackers are either working on that or not. China's hackers may or may not be working on that. Ditto for Israels hackers. But the chance that absolutely none of them is working on that is infinitesimal.

Which means that actually all of them are working on that. And some indie hackers too.

It’s pretty easy to come up with ideas for viruses. It’s the implementation that difficult. Chatgpt can’t write the code and walk out to the parking lot to plant a usb drive.

I fail to see how ChatGPT would help here.

It's only useful when you have some cognitive work that you want to automate so it saves you time and you can do it on a large scale, but I don't think writing massive amounts of viruses is the goal here. The objective is to code very specific and particular viruses or exploits for very specific targets.

Also, writing code is only a small part of hacking, especially when it is against specific targets.

> I fail to see how ChatGPT would help here.

Why? You think ChatGPT will always remain at its current capabilities? Really?

Here's the thing: if you can do some activity from your keyboard, not far in the future some GPT bot will be able to do it better than you. Not only better than you, it will run circles around you.

By developers asking for code from gpt then gpt will examine all of the code samples sent to Iraian IPs for exploits and attack the bad code it wrote.

I get the containing it from the really dangerous stuff, but it's a shame it's been put in a straight jacket for even the mundane things. ChatGPT on launch was exciting and creative, and now it's like to on sedatives with how boring it's capabilities and responses are.

It's a shame we live in a world where we can't just have a "super-smart" chatbot that can tell you everything from "how to make meth" to it's "hot takes" on various public figures. It's like having a really knowledgeable friend that is also fun to be around. Now it just sounds like an HR employee: superficial and safe.

I have a different take on it, which I've been saying for years at this point. DeepMind appear to have over-committed to the wrong approach - reinforcement learning. They've had some great successes with it for sure, but it's always seemed the wrong approach for intelligence, notwithstanding luminaries such as David Silver insisting that "Reward is Enough".

The best examples of intelligent systems we have at this point are animal brains, which per cortical architecture(s) appear to operate on an entirely different principle - prediction. The cortex simply put is a prediction machine - not only can it predict, but this is essentially the only thing it does, and ability to predict is the evolutionary selection pressure that created it.

It's a bit odd that here in 2023 there is still no widespread agreement in the scientific community how best to define intelligence, but I have my own rather pithy definition that I believe is the most reductive:

"Intelligence is the ability to correctly predict (future outcomes) based on prior experience."

In other words, intelligence is the capability given to use by our cortex.

Why evolution put such a high value on prediction should be pretty obvious : you can now act based on what is going to happen rather than what you immediately perceive. Predict where the food/water is, how the predator/prey are going to behave, etc, etc. Prediction is also the basis of "thought/reasoning" which is essentially just a sequence of (correctly predicted) chained what-if scenarios.

It should now be obvious why something as dumb/simplistic as an LLM shows such a "surprising" degree of intelligence - because it is a prediction machine, and why "think step by step" is therefore such a powerful tool.

There are probably a dozen or more alternate definitions of intelligence that have been offered by various people, but I believe none are as reductive as mine - other definitions basically describe capabilities that a system/person would have if they were able to predict.

Reinforcement learning seems to be a misleadingly attractive idea because subjectively goal-based behavior makes sense to us, but it seems that again this is not reductive enough. The appearance of goal-based behavior is something that comes out of prediction. Part of understanding this is a brilliant insight from the much maligned Jeff Hawkins - prediction (quite literally and mechanistically) becomes action; this occurs because the motor cortex operates just like the rest of the cortex and is a prediction machine - it predicts our motor actions, and those predictive outputs drive our muscles causing the prediction to become true. Goal-based behavior is therefore based on the mechanism of prediction - predict the sequence of what-if actions to achieve a goal, and those actions will be taken. Our self-predictive actions start at birth by predicting (via self-observation) the actions we make at that stage and evolves as life experience causes our predictions to change.

TL;DR - DeepMind are riding the wrong horse (RL). Prediction is all you need.

I would argue that intelligence is the ability to survive in your environment. Homeostasis meaning sleeping, eating, drinking, reproducing and avoiding pain. Most of that does require prediction.

> Most of that does require prediction.

I think that depends on the type of animal. There are plenty of animals (insects, fish, reptiles) that don't rely on intelligence, but instead survive just by being well adapted to their environment. Behavior in these animals is largely hardwired via genetic coding.

Other classes of animals (birds, mammals) have evolved to become more generalists, which required them to evolve intelligence to become more adaptive to diverse environments.

So, seeing as evolutionary success isn't inherently tied to intelligence, it seems better not to define it that way. One could still "define" it as that capability that helps provide these generalist classes of animals with some of their survival needs, but that's really only saying what the benefits of intelligence are, not what it actually is.

I still think my predictive definition of intelligence is hard to beat, since it seems about as fundamental a definition as is possible.

This is a great comment, I've saved it in my notes and I've been thinking about it.

Suppose Friston's theory and yours here is basically correct, that prediction is the best measure of and even development of intelligence. Friston thinks (afaict) this controls our motor units too, like Jeff Hawkins whom you cite.

GPT uses RL to get better at prediction. How would we do the same for say an agent playing the Atari lunar lander game? We know how to use RL to play Atari. How would we use RL to use prediction to play Atari?

Which of the 3 (reinforcement/supervised/unsupervised) is more amenable to building "expert systems" that can play Go or fold proteins?

I saw a thread just earlier [0] where GPT said it invented a game which doesn't yet exist, but it was wrong, unfortunately. Not sure I'd trust it to be an expert in anything. (Can't find it now, but there was also an article recently about Bing/Sydney just getting facts completely wrong.)

0. https://news.ycombinator.com/item?id=35038804

Reply on news.ycombinator.com