I keep a pretty sparse toolset. vim+screen and always use them together even on my laptop.

So when I SSH into a box, I pretty much have my "full" toolset just as I would locally. For me, this is important.

That coupled with the fact I have my vimrc memorized, I'm pretty much good on all modern servers.