For the last few years, I've been "seeing maps" whenever I think about LLMs. It's always felt like the most natural way to understand what is going on.

It's also for this reason that I think new knowledge is discoverable from with in LLMs.

I imagine having a topographic map of some island that has only been explored partially by humans. But if I know the surrounding topography, I can make pretty accurate guesses about the areas I haven't been. And I think the same thing can be applied to certain areas of human knowledge, especially when represented as text or symbolically.