Of course the threat model is kinda skewed because this case is more applicable when one's reusing passwords or using weak passwords, which shouldn't be happening if you're using a password manager.
Maybe a more relevant threat is password gets compromised from a MITM attack, in which case they still don't have access to your TOTP