Those 4k livestreams have a latency measured in seconds. We have made massive advances in recent years and if you have good bandwidth you can now have 4k livestreams with maybe 2-4 seconds of latency. But for moving a mouse that's still one to two orders of magnitude too much.

If you allow less latency you can't do as good a job at video encoding, and can't allow for any buffering (client side or encoder side) to smooth out bandwidth fluctuations or allow for more dynamic bitrates. That drastically lowers the achievable quality.

Not that it isn't an underdeveloped area, but the needs of the streaming cases are different. One key difference is if there is no feedback/interaction on the livestream, then there could easily be a relatively long time buffer that would not be noticed. Vs moving a mouse pointer around on a desktop where it would get annoying at a few hundred milliseconds. It also leads to differences in how the encoders and decoders need to be applied.

the video is pre-cached by your ISP