Hacker News Reader

Comment by pclmulqdq | Hacker News Reader

That is amazing and incredible, but still won't make up for the fact that rocm is a garbage fire and opencl is slow. Nvidia has more than good hardware.

View on HN

Arch is packaging ROCm natively in testing, and PyTorch has official-ish builds of it now (and maybe Arch will build it too).

But you should also check out MLIR which, for instance, is used very sucessfully here and is benching way faster than rocm: https://github.com/nod-ai/SHARK/

Debian also has a lot of ROCm related packages from base library to software packages. Looks like AMD is in a push for making it widely available and easy to work with.

I think we should clarify that this push is for making it easy to use, not to develop. They are trying to capture the python-based ML community, but care a lot less about people who actually want to write code in ROCm than Nvidia cares about CUDA coders.

And the datacenter cards (which are even less accessible than an A100) are still the first class citizens, while CUDA performance on the desktop cards is quite excellent.

RoCM works okay for me in casual use, what issues do you have?

ROCm's software stack runs on what feels like Ubuntu LTS only (good luck on Debian or derivatives!), you need very specific PyTorch versions that work with ROCm (no running the bleeding edge easily), and running an old UEFI version (eg: 2 or more years old) often causes a no boot/no POST situation.

ROCm is a fragile stack that works on a very small subset of AMD's hardware with 3 blessed Linux distros. It doesn't have to be this way, most APUs and GPUs from AMD should be able to run ROCm even on Debian, Nix, or a distro of the user's choosing.

Thankfully some Instinct hardware is super underpriced, so if your willing to make the effort you can run things really fast for rather cheap :D

And God help you if you want to program in actual ROCm rather than using a library like PyTorch or BLAS. There are basically no good manuals or usage guides, and the accompanying tooling is much harder to use than the CUDA equivalents (if they exist at all). On top of that, nobody has any examples outside of major libraries or HPC code (which were largely written by developers at AMD).

Where did you buy the Instinct hardware and what's a good price?

When I tried it, ebay was the place to go, and the price was about 1/10th the nvidia equivalent when comparing computing power. You pay for it in time instead.

Maybe comparing raw specs, not accounting for software running slower, if at all. You pay for it in time, but also sanity.

HIP/ROCm has been packaged in debian - all I had to do was `apt install libamdhip64-{5,dev}; pip install torch` and then Stable Diffusion would run, on amdgpu 6600XT, with no third party repos at all.

It should be fine for AMD's packages to only work on their own blessed distros, that's normal behaviour for upstream. Then downstream distros will then repackage it to make it a good citizen in their distro.

Those packages are only in Debian testing, and I think RDNA2 GPUs are the only consumer-level GPUs with reasonably good support in ROCm (and then probably only some of them). AMD also dragged their feet for a long time on supporting RDNA after it launched, and actually managed to break support for the only consumer GPUs that worked reasonably well with ROCm before that (the RX570 and similar) quite a while before they added support for RDNA.

Not discounting any of that, but it seems like a similar situation to early versions of cuda. Those really sucked to install.

I would argue cuda is still painful when you find yourself in some sort of edge case... The thought of using a stack that is worse gives me the creeps.

Debian bookworm will ship with a few bits of ROCm, and there are more things coming

https://qa.debian.org/developer.php?email=debian-ai%40lists.... https://tracker.debian.org/teams/rocm-team/ https://salsa.debian.org/rocm-team https://lists.debian.org/debian-ai/

> runs on what feels like Ubuntu LTS only

It surely runs in the Cray environment (I assume still based on SuSE) on Frontier. There is ROCm packaging in Fedora/EPEL, provided by an AMD employee, but I don't know anything about it.

Are you saying OpenCl is architecturally slow or a particular implementation is slow?

Moritz is able to extract theoretical peak hardware perf out of his OpenCl code.

Reply on news.ycombinator.com