Maybe its a dumb question, but isn't something like OpenCL meant to solve this problem?
Of course this knowledge is superficial and probably outdated, but if I'm not too far off base, it's probably more work to translate a general CUDA-like layer or CUDA libs to OpenCL.
OpenCL is meant for general computation (the C is for "computing") rather than graphics, like CUDA.
In practice, OpenCL became a giant mess. Some vendors put speed bumps by not supporting the transition from 2 to 3, or having shitty drivers for it.
It also sat at the wrong level of abstraction for high performance compute, which is why CUDA ended up being used.
Vulkan would have been reasonable to write compute shaders in, if there wasn't a ton of alternatives out there already now
Well, Nvidia's stock price in the age of AI indicates how bad of a screw up that was, they're locked out of the market growth until they play catch up. By then, Nvidia might have an insurmountable foothold.