Every field has terms of art, and 'reasoning' is one for LLMs. But that doesn't mean it has the same properties as 'reasoning' in other contexts, because you're not referring to the same thing.
Why doesn't my asteroid belt buckle?
Every field has terms of art, and 'reasoning' is one for LLMs. But that doesn't mean it has the same properties as 'reasoning' in other contexts, because you're not referring to the same thing.
Why doesn't my asteroid belt buckle?
> Our findings consistently challenge the prevailing narrative that intermediate tokens constitute a semantically meaningful reasoning process. First, we observe a pronounced lack of correlation between solution correctness and trace validity—models frequently produce invalid reasoning traces even when they arrive at correct solutions. Second, and more strikingly, models trained on corrupted or semantically irrelevant traces achieve performance comparable to, and often exceeding, that of models trained on correct traces, especially on out-of-distribution tasks.
Related: Poster side dialogue and Q&A about this work at ICML. Very good. https://news.ycombinator.com/item?id=49277303
Because the field of artificial "intelligence" is constructed around the idea of applying psychological metaphors to computational systems (a very powerful idea!) it's almost a worst case scenario for these people.
Suddenly, they're reversing the metaphors and applying computational schema to psychological processes ("aren't we really just stochastic parrots ourselves?!"); or, like here, they find themselves surprised and confused when they stumble across the natural boundaries of the metaphor experimentally.
It's because they never had sight of the boundaries in the first place and maybe never can quite see them. The words only make sense to them as literal equivalence, and so their surprise when they run into stuff like this is earnest and deep.
I think you certainly could look at LLMs as "thinking" metaphorically, but I also don't think it is necessarily only a metaphor.
While maybe that boundary can be fuzzy, we're unequivocally and deeply in "metaphor" territory here.