Thinking has to generalise somehow (likely via internal world modeling) or it's just random search with better and better heuristics that you can verify with real word. The heuristic you get is likely why there might be such big differences between humans for example in abstract problems as for baseline survival we are kinda similar as it got pruned to baseline thing that works.
I think LLMs are missing explicit world generation to be "self-contained" but world models + LLMs are basically the same thing that allows this kind of bayesian causality discovery training but maybe less efficient.
Most people for example when thrown some weird before them like cow from tornado while driving will likely just freeze instead of nicely routing around it etc.
It likely hints that it's so much out of distribution that some heavy computational process is starting trying to model what happened or simply there's no good action to chose as you are "unprepared" for this distribution of event at all and likely both are true. I think that's why self driving cars also have problems with these weird edge cases as you need perfect word modeling to know WTF is happening in these edge cases and they simply won't so it has to be very simple heuristic like stop or try not hitting thing whatever it is in the end. We already have good modeling of human and animal behaviour built-in so driving is a bit simpler from computing these aspects at least but IMO perception is really underappreciated how hard it is but recent vision models seems to be approaching human ability.