This is reasonable when you consider that if your computer --- the least secure device you own --- is compromised, your attacker is virtually certain to get your email account with it, because you'll have (at some near point) a logged-in session.
Meanwhile: the #1 concern that laypeople have about security tokens is that they'll be locked out of their account when they lose the token. Authenticator (or Duo) is a perfectly sane answer to that concern.
Finally, it's worth adding that at least the last time I helped someone set this up, you can't remove your phone number as a factor from your Google account until you have TOTP set up. Your phone number is an extremely insecure login factor.
We use, and recommend, Google Authenticator as a backup login factor.
We do not recommend Yubikey 4 keys for normal users. Nerds on HN might get a kick out of them; I say, go ahead and enjoy yourself. We're trying to solve problems for people who aren't computer experts.