It's not about actual nature (the universe etc) being into consideration.
It's about many mathematicians coming to see maths as exploration (physics-style) of a mathematical universe, so to speak, rather than a simple constructive process.
So, they come to see mathematics as a kind of physics in this regard, no in the sense that they concern themselves with the outside nature. But in that math work appears to them as exploring a natural landscape (just one made of patterns and numbers).