I feel like this is missing the for the trees a bit. I don’t think anyone who is saying “we don’t understand how llms work” means what this author claims. The underlying math of transformers isn’t the part we’re talking about.

What people actually mean is we don’t and often can’t understand an entire reasoning chain of a large llm and often can’t trace that back to the specific training data that caused that reasoning.

Anthropic itself had build all kinds of tools to try to understand llm internal “thoughts” https://www.anthropic.com/research/natural-language-autoenco...

I don’t think it’s unreasonable to say “no one understands how llms work.” What they mean is we don’t and can’t follow every one of its internal chains of thought, especially at the pace new models and techniques are being developed.