If you maximize on a 4K monitor and use tmux on a remote server there is a pretty significant amount of latency. I never noticed until I tried alacritty, the difference is astonishing.
That being said, iTerm2 is plenty fast enough for my purposes as it is today.
Dwarf fortress.
VIM, and overall just being fast. It is extremely perceptible, especially if you ever use something like Arch Linux + st (or one of the faster terminals). iTerm is slow as shit compared to that.
Memory consumption and cpu usage will go down noticeably. If you have 20 tabs open it can add up.