I feel like LeCun has been plainly wrong about LLMs. He has been insisting that the stochastic nature of sampling tokens causes a non-zero hallucination property for any given next token such that as output length increases, this will inevitably converge towards garbage.

The reality is that while LLMs can make mistakes mid-output, those interim mistakes don't necessarily detract from the model's final output. We see a version of this all the time with agents as they make tactical mistakes but quickly backtrack and ultimately solve the root problem.

It really felt like LeCun was willing to die on this hill. He continued to argue about really pedantic things like the importance researchers, etc.

I'm glad he's gone and hopeful Meta can actually deliver real AI products for their users with better leadership.