And so far there's no significant entry here for the ARM world. So you can't roll up a server install or Docker container that isn't, fundamentally, a hacked up tarball from some random vendor. And the market doesn't trust that.
As for sweeping anything away, it's true that there's work involved but it's not like we're starting from scratch in 1985. There's a lot of industry experience supporting new hardware and any company serious enough to be bidding on a Microsoft order for a boatload of Azure servers isn't going to walk away because they can't figure out how to package up some drivers.
The 64-bit world is very different. It is very homogeneous, it looks like an x64 server basically. Userspace actually wasn't ported that long ago. Fortunately now all the major Linux players fully support ARM64 servers (you can download an ISO and install on any ARM64 server without any voodoo, just like x64).