I keep a second key as backup for this reason, which honestly is overkill and I only do because I got a second one for free at a conference. Easier solution (which I also use in case I someday need the second one only to discover that the blue smoke leaked out) is to just print out the TOTP secrets and keep them somewhere. I'm usually printing out recovery codes when I get a new TOTP secret so this has never felt like a big deal.
Also easy enough to maintain a keepass[xc] vault for totp secrets, you could keep a separate one from your passwords if you were feeling paranoid. Great support on mobile and desktop for using a keepass db as a TOTP source - and easy to sync with dropbox/email/ssh/your web server/whatever