And they developed this fragmentation by... building good tools, good documentation, and comprehensively supporting them for 15 years in a way that makes people feel safe building on top of them.
It's not fragmentation, they built a moat.
And with their actual understanding of the hardware limitations of GPUs (memory bandwidth) and the parallel work on things like cutlass (if there was ever an unportable thing :-), the coming *Dx libraries (the explosion of cuBLAS/Solver/FFT to allow kernel fusion and new in-kernel linear algebra shenanigans) the slow but steady introduction of sparsity everywhere, I can't see how anyone can but play catch-up.
It's not like other vendors have made meaningful efforts in alternatives. AMD still hasn't released RDNA3 support for ROCm, their open compute platform. Hell, I don't even think RDNA2 has proper support as of now.
There's also the issue of poor documentation and learning material in the wild.
yeah when getting DL up and running on AMD requires using a datacentre card then it's no wonder CUDA is more popular. AMD is enabling ROCm for commercial GPUs now but it's still a pain to get it up and running, because of the inertia that CUDA has.
they are the one vendor who had the insight ~20 years ago to invest long-term in GPUs and have continuously made impressive products while supporting a cross-platform developer base. For this, I reward them with my $$$ (both work and home).