Trusting Trust says everything could be backdoored, but somehow I'm guessing you still use computers.
That's easily a billion dollar motivation right there, and I can't imagine a bunch of low-paid government drones resisting that cash prize. Everyone has a price.
Hence, there's a level of trust that can be gained through observation of failures to abuse backdoors. If they don't exist, they can't be abused. If they exist, then they must be used/abused, otherwise what's the point? Such usage will be eventually discovered. E.g.: The use of the Dual_EC_DRBG back-door to tap into Juniper VPN connections by the Chinese government was discovered and made public.
Of the people I know that work with highly privileged materials, none would take advantage or abuse something like this, even with such a high payout. Even if they did, how would they continue to live comfortably? That said, it just takes one person under the right circumstances to act maliciously, which is why screening and compartmentalization is critically important for these organizations.
But by the time Dual_EC_DRBG was published, we already had alternatives that were better in just about every way, including being much less likely to contain a backdoor.