If you market your hammer as an all purpose tool, yes. If LLM are worse than humans at predicting data it is valid to wonder why, you could think they would be better at it since they are next token predictors.

The problem it helps solve is our lack of understanding of how the models work.

The way we got these models in the first place is by people researching to try to understand how the relevant systems worked. There’s a great deal we don’t know about how LLMs work. This paper is exploring one aspect of the limitations of LLMs, which helps to understand them better - their capabilities and limitations, and the underlying causes.

This kind of work is exactly what’s needed to develop better models - not just LLMs.