It's not a video game. I type programs in it. 30-110 ms is more than fast enough.
Anyway that being said I have some collegues that are not bothered by lagging mouse cursors and other janky stuff so I can somewhat understand that this might not matter to some people.
edit: I just tried out st again on my gnu/linux machine and it seems to be fine with respect to input latency so I guess its not all too bad of a terminal emulator. Maybe it only sucks on macOS.