Hacker News Reader

Comment by rcleveng | Hacker News Reader

I don't get why folks keep saying x86 linux servers here for AI, if anything it'd be M series arm based servers, either running linux or macos. Realistically I'd imagine a set of scaled up mac mini arm servers for running inference or fine tuning on them as more likely being the "ai servers" than x86 based anything. Power is the key thing that they'll be optimizing for, and that's where ARM shines.

View on HN

ARM is not a GPU architecture, nor is Apple Silicon the most power efficient GPU/CPU available to datacenters. The factor Apple is optimizing for is their own profit - they're terrified by the notion that another company might dictate their software margins for them.

Don't they need gpus (for training)? Apple already did a footshoot wrt gpus in the apple ecosystem. unless they have some sort of apple-internal ai chips ready to train models.

Apple's Private Cloud Compute is on racks of M4 chips which have NPUs and GPUs on-die and unified memory access to however much RAM they want to put on them. All of a sudden they're competitive with NVIDIA, but they don't let anyone else use that platform.

> All of a sudden they're competitive with NVIDIA

Apple has no interconnect technology comparable to what Nvidia ships to datacenters. The larger Nvidia clusters measure their addressable memory in terabytes, the value of Unified Memory at that scale is practically negligible (if not wasted bandwidth).

You're making some pretty handwavy generalizations here without a solid grasp on why Nvidia dominates GPGPU compute.

True, all I have to go on is what Apple presented about their proprietary system.

Reply on news.ycombinator.com