There's no normal workload under which Vim should ever slow down noticeably. I routinely have long-lived Vim instances with hundreds of open buffers, and some of the files can be quite healthily sized (tens of thousands of lines). It should even be fine for "pathological" cases (multi-megabyte log files, files with stupidly long lines etc).