Yeah, but now the comparison for many companies (e.g. R&D dept. is dabbling a bit in machine learning) becomes "buy one big box with 4x RTX 3090 for ~$10k and spin up VMs on that as needed", versus the cloud bill. Previously the cost of owning physical hardware with that capability would be a lot higher.

This has the potential to challenge the cloud case for sporadic GPU use, since cloud vendors cannot buy RTX cards. But it would require that the tooling becomes simple to use and reliable.

Certainly, and both AWS, GCP and Azure even on CPU are much beyond simply hardware cost - there are hosts that are 2-3x cheaper for most uses with equivalent hardware resources.