I know people do that, what with cloud based VMs and all, but still... I don't get it.
It isn't as secure as a VPN (and not as convenient), but definitely a stop-gap if you don't want to pay for Client VPN.
On the contrary, most critical software is plenty new - things like MS Office. Still bound to Windows.
The remaining systems rely on truly custom software and should be either airgapped (so no RDP) or rewritten. I'm thinking industrial - they should've planned for this many years beforehand. There were instances back when Windows XP was the main driver.