In "secure" places, HSM's are basically in a vault with armed guards.
I have seen HSMs in a concrete fireproof room, inside another concrete room, inside a regular room, inside a nondescript building. This one required the use of 3 persons (one to authorize entry past the 2 guys with rifles, one to unlock the vault rack with the hsm, another to authenticate to the HSM to perform work).
Company or agency?
a very large private company
HSMs are designed to resist exporting private key material. And to sign something, you normally have to enter a PIN or a password. :)