But the chain of thought as text can be elusive, no? It might not be related with the actual "reasoning" of the model. There was a paper about this.
The basic heuristic of the argument comes from biological organisms. Multicellular organisms are illegible but the overall organism still operates in its environment just fine w/o making its internal communications legible.