Math is nothing like a map -- maps are approximations of something real and they don't have any kind of internal consistency or complexity.
But there's a good argument that math is the fundamental nature of the universe, and mathematical discoveries lead to predictions of real-world behavior. While maps don't predict a thing.
The philosophical discussion isn't around whether math is useful for tracing the arc of a ball in the air, for which it always will be merely a useful approximation. It's more around math as the language of the universe, in things like quantum physics -- there's no "approximation" here, it's more the nature of reality itself.
And here, the philosophical questions around whether our descriptions of quantum physics are "invented" or "discovered" go quite deep, and necessarily involve the nature of human knowledge itself. For many people, these don't "miss the point" at all -- they're some of the deepest, most profoundly meaningful questions that exist.