Can I also interpret this as: 'AMD's pytorch support is so abysmal that inference is 10x slower than it should be'?
You also have to keep in mind some latest gen AMD GPUs don’t even officially support ROCm on Linux. That’s absurd.
AMD has a choice to invest more staff into ML support, they’re choosing not to.
But, it's worth noting there's different levels of "support." With ROCm 5.6, the 7900XT and 7900XTX RDNA3 cards, while not "officially" supported are gfx1100, which have rocBLAS/MIOpen kernels and work w/o jiggerpokery w/ PyTorch nightly and various HIPified inferencers I've tried like ExLlama.