That's just flat out not true. In fact I would hazard a guess that the reverse is true (0.1% of math papers are incomprehensible to most). Not to say that math is simple but that just because something is highly specialized does not mean it is incomprehensible. Of course my only evidence is that the vast majority of specialized physics papers that I read only require some background in quantum mechanics, statistical mechanics and some of the problem domain. I read some papers in math journals too and feel like they're about on par in general.
There are certainly some mathematicians that publish in super highly specialized fields that are remarkably difficult to understand but these are a teensy tiny minority.