Same here. For me, it is the understanding that every bit of this 'convenience' is adding overhead and latency to what should ultimately just be a text editor with some degree of real-time feedback. I can barely tolerate RDP on a local LAN setup for writing code. Across the internet is a joke, especially if some corporate VPN is in the middle.
Remote tools are totally acceptable when a human isn't in the loop on every frame update. The laws of physics are never going to give us a win-win here. If you want remote tooling + cake (i.e., a low-latency UI experience that doesn't live local), you will need to start deploying this stuff closer to the users' physical locations.