Hacker News Reader

Comment by burning_hamster | Hacker News Reader

I think the title of the paper makes more sense if you consider that ten years ago, someone could have written a paper in a similar spirit with a different take on "what is enough". Back then, it would probably have been titled: "Backpropagation of errors is enough".

The last ten years have shown that backpropagation -- while a crucial component -- is not enough. Personally, I would not be shocked to find out in the next ten years that reinforcement learning is not enough for an AGI (as there are aspects like one-shot learning, forgetting, sleep, and other phenomena for which the RL framework seems not a natural fit).

View on HN

Ten years ago we didn't even have AlexNet; I think most people would have thought a paper like that was nuts at the time. The ten years since are what popularized backpropagation as a path to general intelligence. Who ten years ago would have seriously predicted GPT-3? The odd few that did are certainly not the people I would expect to have been dissuaded! And if there's any actual experimental evidence that backpropagation is not enough, I haven't seen it.

Backpropagation was the model for AGI in the 1980s if not earlier. Of course computing power made it impossible for anything to actually deliver AGI.

RL can forget, just start training it on a dataset that is different from what it was originally trained on.

You are right; I should have been more specific. RL does forget in the simplest sense, i.e. that certain weights in your model drift away if the data distribution is non-stationary. Humans seem to be a bit more targeted in their forgetting.

Why are forgetting and sleep relevant? If someone invented a pill that gave you a perfect memory and removed the need to sleep, would you stop being generally intelligent if you took it?

Because they are strongly associated with all known examples of generalized intelligence. Why wouldn’t they be relevant?

> Because they are strongly associated with all known examples of generalized intelligence

Correlation != Causation. While they very likely might be relevant, I've not seen anything to conclusively prove that it is. The ability to forget is important to humans because we are emotional beings, but I don't think that necessarily is a requirement for generalized intelligence. "sleep" (as in what happens during sleep, not the act specifically itself) on the other hand is very likely important, but again, not proven.

I completely agree that there is no conclusive evidence. However, it is the number one activity that influences cognitive performance in all animals that have been tested. Saying that sleep is not proven to be important for intelligence because the causal link has not been established seems a bit like saying exercise has nothing to do with muscle growth, because the full causal chain has also not been established (we do know a pretty full causal chain in this case, so maybe it is not the best example). I think if you were a betting man, and as a scientist you have to be to some degree, you would put your money on sleep supporting some essential process for intelligence.

Possibly. Database look up on a million rows is very different then a lookup on a trillion. Both have solutions, but the Perl hack that is our mind may lock up on a bigger data set.

One of the postulated reasons for why older people have worse reaction times and think slower than their younger counterparts is that the neural networks they use draw upon more stored information, thus making routine evaluations take longer.

There's a sweet spot between knowing enough and knowing little enough so that you get the right answer and get it quickly enough.

That's a weird claim. Why not just assume old people are slower minded for the same reason they are physically slower: physical degredation?

I guess you haven't heard of ISRIB (integrated stress response inhibitor). This drug when given to mice causes both old and alzheimer's riddled mouse brains to return to near normal. Interestingly the patent is now held by an ALPHABET company. Not saying there isn't a limit to our storage capacity, but this drug make it clear that age alone doesn't prevent the brain from working well.

Well, if the state of the world changes, then hanging on what you learned in the past can cause you to do the wrong thing. Sure, there is an old proof that the value of (true) information is greater than 0, you could say, but they could also remember that the state of the world has changed, so there is nothing bad about remembering, or the model could just discount data by how old it is, etc. All true. But the representation becomes more and more complex. I certainly find that I have to pull back and tell myself, wait, the world has probably changed since I learned that, hasn't it? Has it?

Probably because if we didn't have the ability to forget and turn off often we would pretty quickly kill ourselves to end the horror of our existence?

Maybe not, Chesterton's Fence applies: https://wiki.lesswrong.com/wiki/Chesterton%27s_Fence

Reply on news.ycombinator.com