how is a physical yubikey any different from a loss POV?
Also Google relying on Google for anything important/critical is a bad idea. The internet is full of people complaining that Google deactivated their account for no apparent reason.
I guess I’d recommend enrolling both your phone and a Yubikey as a 2FA.