Exactly, your phone can break or get stolen any time. Plus I just don't want to limit myself to a single device.
If you need to manage non-trivial amounts of money through your phone, having a specific device to do that is a no-brainer.
https://www.bbc.com/news/articles/cy8y70pvz92o.amp
I'm not sure exactly how they get around security features, perhaps by social engineering customer support, if they have enough PII.
Obviously people can still kidnap you and torture you but that's no different from before smartphones.