I find that in actual operation Emacs is faster and that's the reason I switched from Vim to Emacs. Even after using all the tricks in Vim's :help tex-slow, Vim still lagged when I scrolled or even typed into LaTeX files (unless I turned off syntax highlighting), and I'm not such a great typist. Emacs is perfectly snappy.
But yes, I admit that I find silly that I should need a 'server' for a text editor. Not to mention that I remote into various VMs regularly which would also need to get their own server/client setup if I wanted a homogeneous working environment.
I use emacs in a work environment that is primarily Vim and where we end up logging in to a lot of machines to do development on, and the two things that turn heads in my emacs setups are my fluent and flexible use of multiple windows and frames, and the fact that I'm always using my local emacs and I use Tramp to remote in to the machines, so I don't have to screw around with .vimrc files on half-a-dozen machines. Both are stock emacs.
But moving to Emacs had many other benefits, so I'm definitely glad I did it. The LaTeX is just what made me look for other editors, being the one thing that really bothered me about Vim.
And about the remote VMs: as user jerf said, you only run one Emacs, on a local machine, and use Tramp [1] to edit remote files as if they were local. It's great!
(package-install solarized-theme)
(load-theme 'solarized-dark t)
That's it.