The answer is: we are humans, doing human things within the scope of human capabilities. If there are ideas that are inexpressible by us, we can't possibly know about them. If there is physics profoundly beyond our ken (what lies behind the quantum veil, for example) we simply don't know about it.
Mathematics is a natural language (as physicists use it) to describe nature to ourselves. The fact of the knowing subject, and the activity of the knowing subject, cannot be left out without leaving a central mystery, which always amounts to "Why does the knowing subject do what they do?" (like restrict math to Smolin's four key categories of number, geometry, algebra and logic). If you imbue some mystical subject-free "mathematics" with these properties, rather than the activity of the knowing subject with them, they will remain mysterious.
This also explains why mathematics is so very, very bad at describing reality: http://www.tjradcliffe.com/?p=381