The HSM market makes me really sad. I'm kind of considering doing an "open source HSM" -- something using COTS components, ideally a few generations old, with everything totally verifiable by the user/assembler, and a key loadable at manufacture time -- the idea being companies could assemble and certify their own, or a third party or industry association could certify them for their regulees. Goal would be to price a low end version around $100-200, and a high-end (x86-64 performance) in the low thousands, unlike the rape that is Thales or SafeNet ($20k+ for mediocre performance). Plus, if you're at all "interesting", there's no way you'd trust a European defense contractor or a US defense-affiliated company with a "black box for keys, trust us".
Not sure if this would be a good business, though. Would need to raise actual VC for it, and while I know some awesome HW people, including in the tamper-resistance space, I've never done volume production of any hardware myself.