Let's look at the simplest D, that is D = d/dx. What function satisfies du/dx = ku? An exponential, of course, because functions of the type u = a^x have derivative u' = ka^x for some k. What'e the most fundamental such function? Well, it's the one where du/dx = u, i.e. k = 1. This is none other than the exponential function e^x with e as currently defined. Every other choice of a "fundamental" exponential function will run into a lot of inconveniences later on.
What about another example. Let's look at D = d^2/dx^2. It's simple enough. What functions satisfy d^2u/dx^2 = ku. It appears the solution depends on whether k > 0 or k < 0. If k > 0, we can show that the hyperbolic trigonometric functions do the trick, so we can rehash the discussion above. If k < 0, however, we get into the usual trigonometric functions, namely what we now know as sin(rx) and cos(rx) for some choice of r, since for both u'' = -r^2u and you can find r from there. Again which r is the most fundamental? r = 1 of course, and this leads to the usual trigonometric functions.
If you choose r to be something else, we'd have a problem as we'll no longer have the nice property that sin'(x) = cos'(x) and so on. Also, these two examples are really the same example since e^(ix) = cos(x) + i*sin(x).
As a side note that should really be the main note...aesthetic calls are very important in mathematics. No one likes scaling factors to their identities. We like using fewer symbols to talk about common concepts. If we could make e = 1 or pi = 1 without inserting the opposite problem of complicating almost every single identity in mathematics needlessly, we would have done it a long time ago.
I think the reason c = 1 works in physics is because you can change a few other constants around (usually to 1) and things works out well. I fail to see how we can do the same in mathematics where things need to work on many many different layers. Also, if you look at any "Natural units" systems, they all have pi in them. Why didn't they get rid of it? Because it's even more fundamental than c!