Until there's a YubiKey 4C nano, I'll wait. Having something of that size sticking out of my computer is not really practical. Not having it inserted defeats the whole point.
How so? I would keep this on a keychain or lanyard (it looks rugged enough to handle that sort of environment). When I need to authenticate, I plug it in, when I'm done, I unplug it. That seems a lot more secure than leaving it in the computer all the time. If someone gets my computer and the YubiKey is always installed, that sort of defeats the purpose of having a separate hardware security device. In that case, why not just use Keypass?
When I leave my computer out of my eyes it usually locked (unlock with password), so I don't expect somebody to quickly go in and do something. If I leave for longer, I sometimes pull the Nano out and put it somewhere I don't lose it for a while.
The advantage of your system is that you can use the U2F screen unlock instead of the password.
Take the devices you use: how many exploits have there been in the last year where an attacker who could get you to click on a link, view an image, etc. could run code on the device? Maybe you have something like the iOS sandboxing between application which would stop a compromised browser from compromising your authenticator app but there are many cases where attackers have been able to bypass that.
Using a hardware token prevents almost all of those attacks and means that if you are compromised you'll have an easier time regaining control. They also have some nice benefits such as not running out of battery when you need them in an emergency.
This thing might protect from keyloggers but useless against proper malware that just waits for you to authenticate.