I agree that U2F is generally a more secure 2FA solution than soft tokens on a phone. If you use U2F (hardware), then what are you storing in your password manager (software)? Maybe, the 2FA secrets for sites that don't support U2F yet? If so, care to share which sites you use frequently that do not yet support U2F?

Also, which password manager do you use? I ask specifically because if it is a popular commercial option (1password, LastPass, Dashlane), then your attack vector on your password vault is larger than just local malware. It also includes remote attack directly on the servers of the commercial company. Sure, your vault is encrypted on their servers, but if a hacker gets their hands on your encrypted vault via malware on your system or via their remote servers its the same outcome: they have your encrypted vault and can start to work on it. Make sure that your vault password is really strong so that it cannot realistically be brute forced (I'm sure you already know that).