Maybe the problem was with your specific setup or applications? Because at the observatory it worked flawlessly. Between the local data reduction machine (beefy server) and the desktop computer in my office the same. And I used that setup for years and it worked just fine (and I really really hate any lag or glitches).

VNC really sucked on the other hand, not being able to transparently share single windows (at least I never figured out how to), some windows would fail to refresh and I would need to drag them around to get them to redraw, copy and paste was always a pain, sometimes inputs not registering properly.

To be fair to VNC though, over the internet plain X11 forwarding really sucked (latency is the real killer here) and VNC won out there. Unless one was using NX proxy, then it blew VNC out of the water (while using X11 on both ends). Only RDP was somewhat on par with NX over the internet, but locally still beaten by X11.