It's not worth it.
I don't run a business. I'm not a celebrity. I don't keep confidential information in my e-mail. And I don't register all of my various accounts at sites around the web to just one e-mail address. If someone got access to one of my e-mail accounts it would probably be a general-use one, and quite honestly it wouldn't affect me much.
It's also annoying to have to verify every time I log in. I probably log in more often than most people. When my browser session ends, all my cookies, cache and history are erased (not because i'm paranoid, but to help guard against CookieMonster, history probing and similar attacks on sites that don't do secure browsing right). Even though it may only be occasionally, having to go find my phone to authenticate the login takes away from my browsing experience, and I don't find the tradeoff worth the hassle.
It doesn't help that Google's authenticator medium is SMS. As a hacker, I find there's way too many avenues to intercept the token (not the least of which is an already-logged-in Google Voice session!). I like the idea of using a YubiKey, but if I have to stick the device into my computer, it's annoying. I prefer plain-old tokens like RSA's SecurID or the PayPal token (I got mine when it was still only $5). But it's not automatic. I have to do a bunch of work to set it up, and i'm lazy.
At the end of the day, even once you set up two factor, a good attacker will still get past it if they really want to. Separation of accounts will go a lot farther towards keeping your digital self safe than putting all your eggs into a two-factor single-account Google basket.
Edit: This post has encouraged me to pay the $20 for Bank of America's SafePass Card, which I consider to be much more secure than an app or sms.