Company mode is the big offender for me, it handles very large completion lists poorly. I'm not yet fully satisfied with the latency situation in my emacs config, but for me there is no going back. Evil mode let me keep almost all of the muscle memory I ever learned in vim/neovim, and the rest of the emacs ecosystem more than makes up for the latency issues in my opinion.