Yes, you can make perfectly valid mathematical systems that have zero anchoring to any physical reality we experience (such as, trivially, n-dimensional geometries).

A geology that isn't anchored to our physical reality seems intrinsically invalid.

In our universe there are some number of physical constants that cannot be determined from maths alone, but only measured. If you go about changing these physical constants then we don't have physics as we know it (change 1/137 to 1/140 and electromagnetics no longer works). You get some totally different physics of which there may be nothing more complicated than hydrogen, or maybe hydrogen doesn't even exist at all.

https://en.wikipedia.org/wiki/Fine-structure_constant

One I always go back to: You can represent a perfect impulse (think electrical signals, a switch from 0 (low voltage) to 1 (high voltage)) mathematically, but it's impossible to physically create.