> I (symmetrically) encrypt my private key

That's basically what a passphrase does already.