Hacker News Reader

Comment by martinald | Hacker News Reader

It's absolutely mindboggling to me that AMD is still struggling so badly on this.

There is an absolutely enormous market for AMD GPUs for this, but they seem to be completely stuck on how to build a developer ecosystem.

Why aren't AMD throwing as many developers as possible submitting PRs for the open source LLM effort adding ROCm support, for example?

It would give AMD real world insights to the problems with their drivers and SDKs as well, which are incredibly numerous.

People would be willing to overlook a huge amount of jank for cheap(er) cards with large VRAM configurations. I don't think they when need to be particularly fast, just have the VRAM needed, which I'm sure AMD could put specialist cards together for.

View on HN

Historically they believed that "the community" would address broader ML software support. I think the idea was they could assign dedicated engineers for bigger customers and together that was a sort of Pareto-goodish solution given their constraints as a company. Even in retrospect I'm not sure if that was a good call or not.

I mean, they would be right if all their cards, both consumer and enterprises, supported the same programming interface.

You cannot trust the community to do the work for you but then only make the software available for $Xk dollar cards

That's not necessary or sufficient. Going back to 2017 or so when I was working in the area their OpenCL support was good enough, the missing parts were an equivalent to cuDNN and upstreamed support in TensorFlow etc. That work does not subdivide in a way amenable to being a community effort and it's way too big for a hobby project. Today the technical landscape is different but from what I can tell the basic problems are the same.

In 2017, when I got Vega, OpenCL didn't work yet.

Today, in 2023, Vega is already not supported.

Meanwhile, during this period, ROCm was unbuildable by mere mortal distribution maintainers; you either used the binaries thrown over the wall by AMD (for specific versions RHEL/Centos, SuSE and Ubuntu only), or didn't run anything at all.

Also in 2023, Fedora managed to package basic ROCm packages for Fedora 38; I can finally run darktable and blender (but it is crashing!) on Vega. Woohoo!

Support for Gentoo existed for a long time in https://github.com/justxi/rocm before being merged in the main Portage tree.

By existed you mean maybe built and sometimes worked.

That's not support. That's throwing things over the wall. More importantly, even with Vega it had numerous crashes. Anything else, like Polaris or RDNA? Forget about it. Even the AMD docker setups weren't quite good enough at times.

No I know, but pretending it was so hard to build it could be considered as good as closed sourced isn't needed to shit on it.

It is hard to build.

AMD uses cpack deb/rpm generators; and the build process requires random things in path (some built from git checkouts of other projects). If you want to create standard deb or rpm build script for building in distro build infra, or inside mock, for rpm-based distributions, the cmake build actively makes steps to make it as difficult as possible.

There used to be a talk titled something like "How to make distribution maintainers hate you" (I cannot find a link to in now); it seems that ROCm developers have seen it, took it to their hearts and then wrote several new chapters themselves.

That's building. Do not even think about testing.

There's a reason why it took distributions years to package it (still not done completely). The upstream project was like student projects, thrown over the wall once students stopped working at it, including the build "system".

We didn't use ROCm, the non-ROCm OpenCL path worked fine for us on Polaris and Vega. None of this is a major reason AMD cards are vastly inferior for ML dev and research purposes. At the time they made a decision not to invest heavily in ML framework and workflow support, and so they never had a product really usable for those applications.

I find it annoying they didn't do more but I'm not sure they were wrong. AMD managed to tread water in GPUs, integrate Xilinx, and go from meh to a very strong position on CPUs, and all of that with a relatively small company.

>People would be willing to overlook a huge amount of jank for cheap(er) cards with large VRAM configurations.

The older I get, the more intolerable I find jank to be because my time only keeps becoming ever more valuable.

Intel has managed to get their drivers on 23.04 Ubuntu with no additional packages needed to be installed for their Arc dGPU offerings.

Now if only they would offer some bigger Arc GPUs...

I would have picked up a 32GB+ Arc over my 3090 in a heartbeat. Maybe even a 24GB card.

16 GB is a really nice offering at that price point for AI workloads. I'm keeping my fingers crossed for a higher end Battlemage offering and some real competition for Nvidia.

They also lag on the dataplane side do they not? AFAIR nvidia bought the main (remaining?) infiniband supplier and seamlessly integrated it with all their data center offerings? Cue Jensen Huang "the data center is the computer"?

They only care about selling data center cards for GPGPU.

The thing is, why would anyone buy them if CUDA just works?

Reply on news.ycombinator.com