Hacker News Reader

Comment by latchkey | Hacker News Reader

Today's MI300x announcement. Worth a watch before you comment about how their software sucks.

https://www.youtube.com/watch?v=tfSZqjxsr0M

I'll add some highlights for those who don't have the time - Flash Attention 2 kernels, Paged Attention kernels, PyTorch, vLLM, transformers, ONNX runtime support, OpenAI triton support in v3, tensorflow, JAX, OpenXLA.

Also, while this wasn't in the presentation, CUDA can be translated to HIP using automated tools. and - CuPBoPAMD is a CUDA translator that translates CUDA programs at NVVM IR level to HIP-compatible IR that can run on AMD GPUs (https://dl.acm.org/doi/pdf/10.1145/3624062.3624185)

> CUDA can be translated to HIP using automated tools.

Not just automated, but also open source.

https://github.com/ROCm-Developer-Tools/HIPIFY

Open source, but only covering some AMD hardware for 4 years. Don't trust the support to last.

One of my coworkers uses a Vega 56 in their local workstation. That card is six years old. It's not officially supported, but it still works. I have no idea if the AI frameworks work on hardware that old, but the math libraries still do.

They've gone so far to be code drop in compatible that an <import cuda> statement is still <import cuda> when running Rocm on AMD!

AMD made a presentation on their AI software strategy at Microsoft Ignite two weeks ago. Worth a watch for the slides and live demo

https://youtu.be/7jqZBTduhAQ?t=61

>CuPBoPAMD

They really couldn't come up with a better name than whatever that is?

Koduri left so they have a chance to make the software not suck anymore

All that buzzword bingo does not mean the software is actually robust and performs well. If you have hardware that has much better specs on paper, but only delivers maybe 20% better performance in their own benchmarks, I am still going to presume that their software is not up to par.

20% is a huge amount in terms of high end performance. For example, 20% more transistors / compute units / etc does not give 20% more performance at the top end unless your execution is excellent.

OP's point is that the hardware specs are 2-3x higher in many places, but all their benchmarks are 20-30% higher. The article mentions this as well. It means AMD couldn't even utilize their own hardware very well at this point.

Transformers are heavily memory bandwidth bound on modern hardware, and these chips only have 60% higher memory bandwidth.

Their own slides show a 20-30% speedup on attention tasks.

20% better performance, lower cost and most important... availability.

Reply on news.ycombinator.com