iTerm2 started doing some GPU rendering stuff in version 3.2 (https://iterm2.com/downloads/stable/iTerm2-3_2_0.changelog). Maybe it's faster since you last used it?
On my 5k display, at full screen resolution, iTerm2 scrolls in a large vim buffer at max 40fps. Alacritty on the other hand is around 60fps. Not sure how to enable an FPS meter on the Terminal app, but it feels closer to 10-15fps.
I used iTerm2 for years, and am still a Patron, but until GPU acceleration gets to 60fps at 5k resolution, I don't think I'll be switching back. It's still one of my favorite open source projects, and is only one of two that I currently pay for each month. So I sincerely recommend it to anyone who doesn't care too much about terminal performance.