Hacker News Reader

Comment by bbor | Hacker News Reader

Can anyone link a credible expert who thinks that LLMs could ever evolve into AGI? Every single source I’ve read said that while LLMs are competent at language and certain tasks, they are conceptually completely distinct from the basic components of an AGI. For example, they lack structured knowledge and metacognition.

View on HN

We still don't know what AGI is or how to ever get there, so you likely won't find a credible expert who thinks that LLMs could ever evolve into AGI but you also aren't likely to find a credible expert who thinks LLMs could never evolve into AGI. Every credible expert that believes in AGI is going to hedge their bets and hem and haw and say that "well it's possible" to both questions (ever or never).

I think that's also why you are seeing a lot of cynical credible experts starting to doubt AGI as a concept or a goal.

John Carmack (ex Oculus, Id software) just had a really good interview about this:

https://dallasinnovates.com/exclusive-qa-john-carmacks-diffe...

To summarise: he thinks we've got most of the pieces now to maybe get within reach of a useful AGI (he likens it to a worker you instantiate in the cloud who can do tasks for you like a colleague) and I think he's saying some of the techniques used to make LLMs could be part of the solution

I think that, almost certainly, the tools that we’ve got from deep learning in this last decade—we’ll be able to ride those to artificial general intelligence.

Calling Carmack a credible expert is a bit rich. He claims to have read a handful of papers. He's yet to do any significant, successful work in the field.

Well no one else was answering bbor's question and I think Carmack is a fairly credible person in tech in general.

He claims to have read a handful of papers.

40 is more than a handful. OpenAI tried to recruit him he asked them to recommend papers:

And he gave me a list of like 40 research papers and said, ‘If you really learn all of these, you’ll know 90% of what matters today.’ And I did.

If Carmack says he studied a paper, I dont take that to mean that he merely scrolled through it while sitting on the loo. I think he's pretty serious. Oculus tech used a lot of AI for visual tracking of hands etc, its not like Carmack is just blundering his way into this. He announced this was his next thing back in 2019

Here's the thing: you don't need AGI to have an alignment problem. It is predicting text, which is surprisingly closely aligned in most cases, but can fail spectacularly. As it becomes more capable and people give it more real-world responsibility, the blast radius for its mistakes increases. That might not be an AGI-level problem, but it is still a big problem.

Reply on news.ycombinator.com