At the end of the day, it's whatever works best for you. I've been using tmux for a year and a half and it's just as important to my workflow as vim. I've had people complain to me that I'm too fast with vim + tmux (either people I'm pairing with, giving a workshop to, or just demoing some code). It works for me. Vim looks great for me. I'm able to think less about what program I'm working in and more about how I get things done faster and with less context-switching.

If you're using GVim or MacVim or some other GUI and it works, wonderful! If you're content with the speed that you're able to code, that's awesome. That said, back when I was using TextMate and Terminal, I thought I was really fast; and I'm probably 30-40% faster in vim than I ever was (or could be) in TextMate.

Clipboard usage is a bit odd; that said, how often do you spend using the clipboard every day? If the time spent mucking with clipboard is less than the time you save doing everything else... then you have your answer as to whether switching will be worth it to you. It absolutely, positively, with out a shred of doubt, is worth it to me.