No. the universe isn't math. The universe is matter. Math is a part of the universe. Therefore math is simply material. Nothing more, nothing less.
So yes. Math is also a material manifestation. Life is a material manifestation. Thought is a material manifestation. The problem is with you trying to separate yourself from matter, but the very thing you're using to create that separation is thought. It is the very thing you're trying to get away from, and so you cannot exist separate from thought by using thought. It is a futile and empty game.
Especially not when they can be independently rediscovered without having any trace of them stored anywhere.
About rediscovering, you can simply argue that ideas are matter properties so an intelligent being can recognize them.