Great point. I think one of the issues with calculus is that it can be studied at so many levels. Most high school classes give lip service to limits (somehow they're important), and then get into symbol manipulation (move the exponent down, subtract 1, etc.).
I think the most important thing in the beginning is to learn to think conceptually about calculus, about how to break a shape into a simpler model and solve it (not just manipulate models that are given to you). Later on (in analysis, which I've never formally taken :) ) the details about the transfer principle, epsilon-delta proofs, etc. can emerge, but realistically only a very, very small fraction of students need that. That fraction can be increased if they are able to use calculus regularly (with infinitesimals, say) and then wonder the details of how it all works.
I like to ask: Would it be such a bad thing if everyone understood calculus as Newton did, at a baseline level? Many people taking calculus leave with no understanding whatsoever, but the same isn't said for geometry or even algebra (you can make references to parallel lines, equations and feel that people will 'get it', but mention a derivative or limit to someone 10 years after calc class and you'll get a blank stare). I advocate having rough mental models that are successively refined over time (vs. presenting the most rigorous version first).