I thought the opposite: if CUDA were easy to replace, wouldn't it have been replaced already?

PyTorch supports both ROCm and Ibex, but it's support is spotty and incomplete to the point where it becomes a pain to use despite AMD spending billions on improving its software layer.

NVIDIA is reaping a decade of design decisions on CUDA compatibility made less sloppily than its competitors.