Most people using software only solutions won't ever have their keys stolen, but that's because nobody tried to steal them. The compromise of a client os is inevitable if targeted by a competent actor, given enough time.
Smartcards and HSM's may not be infallible, but their rate of compromise appears to be negligable at best and an extremely rare capability for an offensive team to have access to.
Smartcards are surprisingly cheap and easy to work with, and due to their simplicity and long history are quite secure. The only real attack on them involves physical access and causes obvious physical damage that'd be impossible to miss.