But if you're that careful, what is the VM really doing for you, and why the hell are you even exposing yourself that much? Just use Lynx or something.
The real solution is this: Install Firefox, install noscript to nuke all javascript, install ublock too, and get a password manager. Selectively allow any webpage interactivity, as necessary. The world isn't a Tom Clancy novel so you don't actually need to do anything more than this to be very secure and on top of almost all active threats.
Ultimately to achieve what we all actually want (strong isolation guarantees that would prevent a full browser exploit from both A) your SSH keys from getting stolen and B) also your gmail spool from being attacked, and let's be honest, B is the worst case scenario) requires a rethinking of the fundamental software stack from OS to user-visible applications. No amount of Browsers-in-a-VM are a substitute.