Hacker News Reader

Comment by est | Hacker News Reader

IMHO, there could be a tipping point of scaling "primate" brain. For a given task, when size of brain can handle so much knowledge that unknowns are fewer than knowns, the advantage grows exponentially. Because you can deduce outcomes correctly by eliminating the wrong answers without actually "understands" the problem itself.

View on HN

Idk. I think the real tipping point is being able to succesfully use external memory so that you don't have to keep everything in your working memory when doing reasoning or solving problems.

Yes, humans have been around for 300,000 years and only in the last 10,000 years human crossed that tipping point to be able to think higher order things.

Do you have any evidence for this claim? Just because people didn’t start agriculture doesn’t mean they weren’t as smart as people today. Hell, 300000 years is nothing from an evolutionary perspective, they were for all practical purposes genetically identical.

It was just most probably a nurture thing — there was not as much accumulated knowledge to “bootstrap” a more advanced civilization (or perhaps, no will to do so. Why change when your system works just fine?). It’s not like Einstein could have achieved much back then without “standing on the shoulder of giants” before him.

That is basically what continuous learning is and so far everything I have seen is that it is discounted as not important or something you could easily work around. There is a difference between waiting for the next minor release which updates the weights based on whatever OpenAI gathered from your chat history and having the model update its weights as you query it.

This is exactly what happened with large language models: they are quite useless until a certain number of parameters, after which their abilities quickly become impressive (read more here: http://arxiv.org/abs/2001.08361)

Except they are fundamentally limited and can’t algorithmically think no matter the scale?

I was merely commenting on the apparent similarity between the phase transition in ability observed in the brain of primates and artificial neural networks, without necessarily equating the capabilities of the two.

On that note, maybe in the future we will discover some neural architectures that are able to "algorithmically think", who knows.

Do you have any literature on this?

Just my own thoughts. Add `IMHO` prefix.

Reply on news.ycombinator.com