So I'm not sure this is a valid critique.
The answer is a lot.
Crypto software implementation vulnerabilities are very common, but the kinds of things you're talking about are most often found in obscure and/or serverside software. Look at the tempo at which bugs like the NSS e=3 bug are released; it's like once or twice a year.
The sorts of bugs I'm talking about exist in client and popular software. As far as tempo is concerned this year alone has given us BERserk, gotofail, Android Master Key, OpenSSL fork(), Bitcoin's use of P256, GNUTLS X.509 parsing bug, the OpenSSL compiler optimization+processor family randomness bug, and others.
If we were to entertain OP's point maybe there would be a faster tempo if the NSA were helping out. :)
But I did also mean that more broader than just construct attacks..., implementations of cryptosystems are often flawed in low level ways which people without special expertise are unlikely to notice... both from a design perspective (any of the great many protocol design flaws in TLS that have turned around an bitten us), or straight forward coding (e.g. it wasn't the NSA that reported reference implementations of Curve25519 had broken carry propagation).