I'm not sure how best to say it.
Implementing primitives & protocols requires little cryptographic knowledge. It does however require significant knowledge about program correctness: testing methods, proofs, and if side channels are important, the characteristics of your platform, and an accurate enough idea how your compiler or interpreter works.
Likewise, to implement an energy constant implementation of Chacah20 in silicon, you don't need a cryptographer, you need a hardware designer. The only thing you need a cryptographer for, is telling the hardware designer to make it constant energy — or convincing the higher ups why the extra cost is justified.
The blog post you link (which I love by the way) seems to confirm my view: many problems are ones of correctness. I believe most such bugs would be caught by corrupting the inputs, as I alluded to. Here, corrupting the password would fail to abort, and you'd catch the bug.