The point Marcus is making is that LLMs are solely next word predictors - intelligence requires far more than that.

LLMs will always have the same basic challenges - their 'kinda copy what a human has written' algorithm has hit its limit.

He is an advocate of symbolic learning - getting that to work is a lot harder, however it's far more likely to result in genuine intelligence.