Not sure why it should be a surprising thing. Senses work together. Not isolated. I associate numbers with taste (2 and 5 are sweeter, 3 is salty, 7 is bitter and so on), and with spatial ladder-like locations (ladder goes up from 1 to 10 and then it becomes like rope bridge until 20 and then its like a hill climb until around 40 and then it is steep down- hill upto 100.
Words are definitely associated closely with geometric shapes, and with touch and sound.
Also, most importantly, all thoughts are attached to places that you have already seen. You can't think about anything without attaching it to a known place. When you forget something, it's because that physical place (room, road, or park) is crowded with other thoughts.