Everyone I know who works in VDI complains about latency. My wife’s whole company uses virtual desktops, and anytime she’s working from and I look over her shoulder I shudder, it’s incredibly noticeable.
Moving windows around is something that I typically utilize to test for lag and there is just none. My previous employer was literally using this for 7 years before I came to my current employer and I was so happy when we switched to VDI.
Of course it depends on your Internet connection and your ISP's connection pathway to get to your VDI but I've regularly connected to my VDI desktops over the years from Starbucks WiFi, Dunkin Donuts wifi and many others with no perceived latency.
inpreceptable latency in teleconferencing doesn't tell you much. There's already so much latency built in (eg. the webcam itself, the encoder/decoder, the time it takes for your packets to travel over the internet to the relay server and then to the other attendants) that the extra latency added by VDI can be buried. On the other hand when you're doing any sort of interactive editing the latency is very noticeable, because there isn't much latency to start with.
Ive been told when my managed device arrives I will no longer need to use either VDI, but if that's not the case I am very seriously considering moving onto another role despite being only ~2 months in.