Yeah, Syntastic is my number one offender. I'd love to see Syntastic be able to just maintain a background server linting process (or several). I can imagine that alone, skipping all the environment loading/code parsing, would likely be good enough to make all but the longest files lint fast enough to be unnoticeable.

Five seconds?! Mine hangs for at most half a second (and I thought that was a lot). I almost don't even notice it anymore.