I use exactly that. It's fine for me and I'm very much a terminal first person.

Maybe people have extremely fancy shell setups? Perhaps it's a font thing that slows it down? Can you demonstrate?

I'm usual a tmux/zsh/vim user (although I've promised to switch to emacs for years :( )

I just timed my xterm. I'm displaying about 2.3MB/s of characters.

Basically here was my test

    $ dd if=/dev/urandom bs=10M count=1 | hexdump -C > /tmp/test
    $ ls -l /tmp/test
    -rw-r--r-- 1 chris chris 51773449 Oct  8 09:32 /tmp/test
    $ time cat /tmp/test

    ... wait wait wait ....

    51773449 / (total time) / (2^20)
   

Now this of course depends on the size of the window. I'm on a 4320x3840 dual 4k monitor and the window geometry was 1334x928 for this test.

I acknowledge this computer (i7-8650U ~ 2018 thinkpad with a boring UHD Graphics 620) is capable of faster output but the xterm featureset is vaaassstt (albeit extremely complicated) and I've found the other terminals to be lacking.

My only faithless move away from xterm was probably 17 or so years ago with rxvt when I was on a laptop with ballpark 16MB of memory. It's memory footprint and startup time was way smaller probably for its total lack of features and unicode support. That made a difference on my Toshiba Protege 120mhz pentium I was using at the time.

When I was a macOS and a Windows developer, slow terminals were definitely a problem though