The claim isn’t that you can’t learn it from text, but rather that this is why models require so much text to train on - because they’re learning the stuff that humans learn from video.
Another issue is what we really care about is scientific reasoning and there, if anything, nature has given an anti-bias, at least at the level of interfacing with facts. People aren't born biased towards learning Metric Tensors and Christoffell Symbols but it takes only a few years at a handful of hours a day using a small number of joules for many humans to get it (I'm counting from all grade school prerequisites vs GPUs watts x time). Much fewer for genius children.
People's brains are wired for language, it's natural we have a head-start.
1. hardwired for creating visual models of 3d / 2d space
2. hardwired for fine-grained hand motor movements.
So the learning is just combining those two skills into the ability to make the brush strokes needed to output the model onto paper.