Right. I think the question though is why isn't somebody fixing this situation? There's money sitting on the table for them when they figure it out and get their act together.
They are. Its called "Use ROCm". Tensorflow support, PyTorch support, etc. etc.
Yeah, its limited to Linux, its limited to a few cards. But within those restrictions, ROCm does work.
Both with AMD MI100 providing the bulk of their compute. Frontier seems like it was given development boards of MI100, because AMD is talking about how they already ported some code over to the MI100 and tested it.
Nvidia has quite a head start. You're not just talking about some simple driver support either. You're talking about runtime compilation/JIT(to target various flavors of HW), tooling support, library optimizations, API stability and maintenance... AMD can catch up, but unless they come up with a new approach it's going to take a long time and a lot of smart people to do so.
I think they will. AMD has the challenger mindset. They rose from the ashes and now actually compete with Intel and they can tackle NVIDIA as well.