Put a key on your phone or an USB stick. I make a key for every machine that I have, and that keys never leaves that machine. In addition I keep another key that I can use to get started again if my laptop gets stolen or breaks.
If I lose the phone, there's a pretty good password on that key that should give me ample time to remove access rights for it.