Hacker News Reader

Comment by FloatArtifact | Hacker News Reader

Well AMD really needs to work on its software stack to support AI.

If AMD throws twice the VRAM onto their cards they'd be a no brainer for that regardless. OpenCL is pretty well supported now.

AMD is trying to get people to pay $5000 for a workstation card too. The days of Radeon VII being $699 are long past.

Part of it is that as you shrink, the PHYs don't shrink much, so everyone is incentivized to minimize the number of memory channels and reduce the PCIe bus size/etc on lower tier products. And in turn, since GDDR6 tops out at 16 Gbit (=2 Gigabyte) per chip, that means a 4-PHY card tops out at 8GB, etc.

And while you can do clamshell... you want to be selling those cards to workstation users, not giving them away to gamers! It is the same problem NVIDIA faces, the fact that GDDR density has not increased leaves them with one single move (clamshell) and they've traditionally reserved that for workstation cards (and 3090) to increase margins.

Rx 7900xt is $760 and it's an incredible bargain now, IMHO ...

Yeah if you don't need CUDA the 7900XT is better right now than people give it credit for. People are super mega butthurt about prices right now, to probably an unreasonable degree, and they're ignoring some of the actually decent options that exist.

4070 at $600 (rip microcenter steam GC deal) is a pretty ok deal too, much better than people give it credit for. For less than a 6950XT you get 4GB less VRAM but it pulls 200W less power which is very noticeable, and gets DLSS2 (even if you don't like DLSS3!) which is significantly better at 1440p and 1080p, which is a big perf and perf/w boost, with better quality than FSR2. Even HUB now likes the 4070 over 6900XT for a generalist kinda build: https://youtu.be/Iy3ikm8MxOM?t=875

Or yes, the 7900XT on top, the launch MSRP sucked but $700-750 is OK for what you get. If you are otherwise getting 6900XT/6950XT I'd probably just recommend spending up and getting the 7900XT (or 4070), they really are a lot more efficient and have better featureset etc. It'll be worth it, suck it up and do it. 6800XT makes sense at like $450, that'd give it space vs the 4070, but people are getting irrational over the whole situation.

3090 is also an underappreciated competitor. What if there was a 3080 Ti 24GB, with DLSS2 and stuff but not $1600? There is, it's $700 on ebay. However, I am uncomfortable with the VRAM on the back with how hot GDDR6X runs, a lot of those cards mined for a lot of years... if you can get 3090 Ti it has only VRAM on the front, or get an evga one, or something. It's ok, at $700 it's a similar proposition to 6900XT/6950XT at $600 but a bit more VRAM and you get DLSS2.

6700/6700XT for $300 is a screamer of a deal and it's not really surprising AMD can't beat it. They're in the "1080 Ti vs 2070" situation, they over-cut on the old stuff and the new stuff can't really edge past it nor is the value great against a deeply cutdown older card (6700XT is a $480 card for $300!). It's not gonna last forever, if that's the featureset you want I'd consider buying. There will probably be a decent replacement eventually but that's clearly the value peak of what they can do with RDNA2 and it makes RDNA3 look poor in many ways. The successor in this price segment is, best case imo, 7600 16GB at $299-329, and that's a bit slower and more limited in a lot of ways, and not really more efficient either. N32 may not really compete favorably with it in either cost or perf/mm2.

4060 Ti is junk but 4060 8GB at $299 for basically 3060 Ti performance (4060 Ti is closer to 3070 at 1080p and 1440p) is reasonable imo, and DLSS will put it clearly over the top of the 7600 at 1080p (FSR sucks). I'm guessing AMD has to get $279 down to $249 or $229 by the time the 4060 launches. I'm guessing 7600 16GB will probably launch at $329 and drop to $299, and that'll be a decent option vs 4060 8GB too (and cost viable). Probably there will be a 7500XT 8GB or 6GB cutdown at $199 (12GB at $250?), doubt they can go too much below that (die cost isn't the problem/cutdowns don't help memory bus size). It'll be a bit slower than a 6700XT for sure, and not all that much more efficient, and you go down to a PCIe x8 bus, but it does have some newer stuff.

I think the 16GB versions of both 4060 and 4060 Ti are DOA, the 4070 is a lot faster and has enough. 6800XT is potentially still compelling in this segment too.

AMD really needs to figure their shit out with the N32 die though. They actually do need a competitor to 4070 besides just RDNA2 rebrands. I think with the unexpectedly (apparently) poor performance of RDNA3 it's just not worth it, like I just haven't heard any rumor mill shit about N32 at all. The MCDs alone would use as much 6nm silicon as a whole 7600 and then you have a chunk of N5P too. With how short wafers are for other products (IO dies, Epyc, N31, etc) it may just not be worth bringing N32 to market. Who knows, but, I'm getting more and more curious.

Numbers: https://www.reddit.com/r/hardware/comments/13vm5ti/geforce_r...

Tim from HUB talks candidly about FSR: https://www.youtube.com/watch?v=ycXkvVfc2yw&t=2541s

(the quality difference at 1080p and 1440p is significant, and that's where $200-300 cards will be running. Even in the 4070 segment... starting to become an issue for 7900XT etc)

I wouldn't call the 3090 at $700 underappreciated. Look at any number of AI projects on github: All numbers you will find mention testing/running the models on one or two 3090s. Among people doing AI stuff it's pretty much the standard consumer card.

ATI drivers have been terrible for over 20 years.

But on Linux they're superior to NVidia's closed source garbage.

How so? You mean graphics or compute?

The whole concept of the way graphics drivers works is bonkers, thanks to IP issues and game patching, but in production the NVIDIA drivers are very reliable and sophisticated.

From my perspective as an admin of several GPU clusters.

You have GPU clusters with AMD devices?

They were at some point but they're better than NVidia now for normal desktop etc use at least on Linux. They went open source and upstreamed the drivers, like Intel.

On the GPGPU stack front it may be different but CUDA is also really low level and abstracted away by ML stacks. And some of them also had OpenCL / SYCL backends at one point, I wonder what's the story there.

Unfortunately, the ROCm stack seems to use a fair bit of separate kernel code and it's not nearly as stable in my experience. I have a 5700 XT (not officially supported by ROCm, but close enough to some other cards that some have had success). It's perfectly stable for demanding games, but just straightforward DMA is enough to cause GPU hangs with ROCm for some reason.

Have you revisited it recently in the last 2-3 years? OpenGL, Vulkan and DX12 drivers have been completely rewritten from scratch and share a common hardware abstraction layer. They pass all Khronos conformance tests, OEM and ISV certification tests and have similar performance as Nvidia on SPEC benchmarks.

Obviously you stopped buying AMD products 4Y ago when Raj Koduri left. Without his corrosive effect, many reviewers are saying AMD adrenaline tools are NOW MUCH BETTER than NGreedia ...

Indeed, are you aware of the current state of AMD's equivalent to CUDA? And how far behind would be, and what would need to be done to near equivalence? It seems like the opportunity and differentiator. I wonder how Apple silicon software is doing...

Instead of trying to integrate the whole stack of, say, pytorch, Apple's primary approach has been converting models to work with Apple's stack.

https://github.com/apple/coremltools

Clearly no one is going to be doing training or even fine tuning on Apple hardware at any scale (it competes at the low end, but at scale you invariably will be using nvidia hardware), but once you have a decent model it's a robust way of using it on Apple devices.

I’ve tried a few models and none have worked. It’s not that they need more resources, just that it freezes and then dies with an inscrutable stack trace somewhere in the OS. If someone from AMD sees the parent comment, please don’t copy Apple!

Mind sharing which ones and your approach? Haven't there been shared posts of success to follow?

Also, yes, anyone from AMD, I think we all want more variety in the market, what do you need help with?

Not every model feature and op is supported (though it is growing with every release), but personally I've had surprising success with it. It has allowed me to leverage some models efficiently on both Intel, Apple Silicon and iPhone/iPad devices.

I mean...the number of people using PyTorch models ported to CoreML is probably several magnitude greater than the number of people actually having success with AMD's initiatives in the space thus far...

Reply on news.ycombinator.com