Wouldn't a 100% Cuda-compatible toolchain suffice? Then you could write your software for Cuda and the software environment would run it on whatever is on your computer.
In many ways it feels like the same situation we had in the 70’s and 80’s personal computers, with many different software bases divided by different hardware implementations. In that regard, Unix was a boon in that it allowed one to write software that’d be compiled out of a single source into multiple architectures (rather than the PC-clone way, where non-identical implementations were driven to extinction).