This is a misunderstanding of what math is, I think. You can invent a perfectly valid mathematical theory that conflicts with observations. Math is just a sequence of "if this, then this" and if the conclusions follow from the premises, it's math. But if you demonstrate that in the universe we occupy, a premise isn't true, then the mathematical theory isn't any less valid, it's just not sound in our universe.
For example, there's a ton of completely valid mathematical work on the correspondence between anti-de Sitter spaces and Conformal Field Theory. However, much of this mathematics has no application in our universe, because our universe seems to be a de Sitter space (positive cosmological constant/expanding), not an anti-de Sitter space (negative cosmological constant/contracting). That doesn't make their math invalid, it just makes it not real.
You can also do a lot of math in Minkowski spaces, which are flat. But our universe isn't flat, it's curved. Doesn't mean it's not math, just that it's not real in our universe.