I know that the real numbers aren't "just notation" because things like $\pi$, e, $\sqrt{2}$ etc can't be expressed using pairs of natural numbers (i.e. as rational numbers). I'm having trouble finding an equally simple example of "can't do this using pairs of reals" justification for complex numbers. Are there any simple and obvious applications that require continuity/calculus or topology on the complex plane that can't be restated in terms of real-pairs (and calculus/topology over those)? Trigonometry arises whenever you are able to take limits over any kind of cyclical/repeating process, so the fact that sin and cos appear in Euler's Identity does not seem terribly profound to me.
Also, why stop at complex numbers, since it's "turtles all the way"? Quarternions, octonions are neat too. Why doesn't Feynman think octonions are "our jewel" too? At each step (real -> complex -> quaternion -> octonion) you get a larger class of formulae having roots, but sacrifice more properties of the ring structure. For example, the reals have a total order but the complex numbers do not.