Five seconds?! Mine hangs for at most half a second (and I thought that was a lot). I almost don't even notice it anymore.
I think it just takes a long time to load up all the gems rubocop relies on, then to process the syntax. I put the blame squarely on Ruby, not vim.