Comment by dagmx | Hacker News Reader

Comment by dagmx | Hacker News Reader

Obviously nobody is competing with NVIDIA right now given their strong CUDA+Mellanox advantage. But that second place is a very important spot.

I think if Apple sold to businesses, it would be really bad news for AMD.

For comparison, take an MI300A which is AMDs H100 competitor. It’s under half the price ($20k) of an H100 (~$30-50k) with more HB3 memory. It maxes out at something like 128GB of memory. I can’t find the price of a GH100 in todays market.

Then there’s Apple. An M2 Ultra Mac Studio is <$7k for 192GB of memory. Granted it’s much slower memory, but that’s not a huge factor for many workloads.

So this M2 Ultra machine is 14% of the H100 and 35% of the MI300A. It has more memory and a faster per-core processor that consumes 40% of the power of the MI300A.

Now of course there aren’t good benchmarks to compare the exact loadouts and configurations here, and like I mentioned the memory is slower LPDDR5 instead of HBM3.

but even then, you can get multiple Mac Studios for the closest competitor. It’ll cost you significantly less to run. The extreme memory bandwidth of HBM3 is not unique to anyone and the price difference would likely more than cover it.

So then there’s the software moat. NVIDIA has CUDA and then after them, Apple seemingly has the widest support of ML libraries. AMD has improved a lot with Rocm but they’re still lagging behind Apple and way behind NVIDIA.

So anyway, if Apple decides to bring this to market, I would be very worried for anyone vying for second place. The biggest issues will be whether macOS is a turnoff (doubt it) and whether Apple can do enterprise (remains to be seen).

> The biggest issues will be whether macOS is a turnoff

Could Asahi Linux become relevant here?

I would think not. The GPU is a bit part of the equation for compute , and there’s very little in the way of ML libraries targeting the standard graphics APIs available on Linux.

Is that true? Linux is the number one choice for training and inferencing deployment, somebody must be doing something right. It's not like OpenAI was training their LLMs using CoreML, Apple isn't even a contender in the game. They were late to supporting basic industry-standard libraries like Tensorflow and Pytorch, and they still don't commit to an Open, cross-platform alternative.

Apple is the #1 loser from the lack of standardization here, and you're trying to make it out as a Linux problem. Imagine if someone from the industry could hear you talking right now.

iOS developer here, CoreML is used for edge deployments in the ML space. Just not training. Obviously linux is king but that's for server side my dude.

"edge deployment" by which you mean client-side application-specific inference. That's not bad, but it's also table-stakes when ONNX/Tensorflow exists and obviously isn't analogous to the server products Nvidia is shipping today. You can "deploy" CoreML the same way Android developers deployed Tensorflow for edge compute 5 years ago, but that wasn't even interesting when people demoed it on Raspberry Pi. Apple is behind.

> Obviously linux is king but that's for server side my dude.

But isn't that what we're talking about? The submission title is "Apple is developing AI chips for data centers".

Right, "developing"

It's a tiny bit ignorant to even compare consumer memory to HBM. Anyone can get 1TB for their server for less than Ultra chips. The reason you go HBM is precisely because of bandwidth needs.

I specifically call out the memory difference though precisely because I know they’re different.

But I do not believe the cost for 192GB of HBM3 + any related changes to the memory system on the chips would be $13k, which is the difference between the Mac Studio and the MI300A.

The mi300x is AMDs h100/200 competitor.

I know. But the M2 Ultra is the MI300A’s competitor. I couldn’t find the pricing for the GH100, but used the H100 as a standin since it’s already so much more expensive to not really affect the point I was making.

Tensorflow seems to run fine on Google TPUs.

Yes but Google don’t sell their hardware to others, so there’s no pricing information to compare it with.

Does Apple ever compete on price?

Yes but only if you shop to match the same criteria.

E.g a MacBook Pro may seem overpriced if you compare to average laptop costs but if you shop within laptops with the same display, battery life and performance unplugged, it is very fairly priced.

Similarly, Vision Pro seems expensive compared to average consumer headsets, but if you match its capabilities (eye tracking, high resolution display, etc) the next competitor are multiples of its price.

Reply on news.ycombinator.com