You'd probably be wrong. There are several consumer accessible chips (i.e. from Atmel that you can order on digikey) that use tamper-proof silicon where the very act of opening up the package to inspect the die destroys the private key data. Even civilian tamper-proof tech is extremely robust, using physical phenomena to protect the data instead of obfuscation.
That doesn't preclude someone with the resources of the NSA from figuring it out or backdooring the design but the resources needed to defeat tamper proof devices is orders of magnitude greater than those needed to use them and that gap is constantly growing.