I think it's considerably more important to have a (basically good) work environment in which your colleagues can sit down and use fluently than to have one that's absolutely optimized to your individual tastes.
When I am not pairing for the majority, I setup my machine to my tastes, and simply install a "default" editor for the few times someone else actually needs to type on my machine. Emacs and IDEA both have good ways to revert everything to defaults temporarily.
I do not believe, however, that a perfectly customized setup has only small gains. For example, consider two tools that I have never seen as defaults in any editor or IDE: rainbow parens/brackets and paredit mode. Rainbow parens just highlights pairs: http://stevelosh.com/media/images/blog/2010/09/rainbow.png
And here is a short minute paredit mode explanation: http://www.youtube.com/watch?v=D6h5dFyyUX0
When actually editing code, those two tools save me several hundred keystrokes an hour. And those are just the first two tools I thought to use as an example out of the entire toolbox that is emacs.
If I was to now give up just those two tools, I would be sentencing myself to the mental equivalent of tying five pound weights to my legs for the marathon that is the rest of my programming career. Why would I intentionally require myself to type more keys per hour to do the same editing, rather than just learning a faster way to edit? This is, obviously, also my argument for learning the vim editing system (which I also use in emacs).
And that's another thing that a standardized environment does: it gets techniques out there for the whole team. Because honestly I don't really believe that all that many people are truly getting their environments perfectly optimized for themselves. I think that most people in a shop where everyone does their own thing end up with high un-optimized environments.
Good environments for all, that everyone can use is better than perfect environments for a few, and mediocre environments for most, plus lack of interoperability.
I'm not sure why my work environment should be optimized for a <1% use case.
My contention is that:
1. Work environments -- including ones that don't formally pair -- should be highly collaborative.
2. It's typically massively less efficient to help someone past a block or communicate a new coding technique via telling them what to do rather than doing it. Much less anything even remotely iterative, where you need to, say, write some code, look at some output, and write some more code.
3. The difference between "a basically good development environment" and "a development environment that is specifically optimized for just specifically you" is actually quite small. There's no real need to sacrifice much of anything in order to get an environment optimized specifically for you.