The whole article is about just that : the LLM has to know what an "animal" and a "street" is to perform effectively.

But then dumbly concludes : oh is language intelligence ? ("Perhaps our form of intelligence is tantamount to language modeling, too?"). C'mon.