Hacker News Reader

Comment by zingelshuher | Hacker News Reader

For those who want to run big models: AGX Orin 64GB is 'just' $2K. Not much processing power, but 64GB of shared memory is quite a lot for 2K.

View on HN

A slightly more expensive Mac Studio is probably a better buy than Orin at this point.

Care to explain why? The Orin option seems massively more scalable and you can put in as beefy a machine as you want without paying the Apple tax which scales exponentially with machine specs.

exponentially is some exponentially scaled hyperbole.

that said, i agree with your question!

Not the OP, but I would suspect the Mac Studio will retain its value better than an application specific product.

You will be able to resell that 192G Mac Studio for 3-4k after a few years easily, which you've bought for 6k. There's nothing competing with Apple Silicon right now for local LLMs when you look at TCO.

I don't understand anything you're saying. Orin is not scalable at all and (IMO) overpriced given its slow CPU and GPU and it's limited to 64GB of soldered RAM while Macs can go up to 192GB. AFAIK Orin is limited to a special build of Ubuntu and has no community.

> Orin is limited to a special build of Ubuntu and has no community.

Not anymore, now they allow a list of distributions. It has NVidia's first class support with all things optimized. Like CUDA, Pytorch, and so on. For now it's the best thing money can buy for mobile robot. I'm interested in Pytorch, CNNs, LLMs, video processing sort of things.

Community is nice, but not sure it's possible to get reasonable performance easily from Mac Studio. Don't remember seeing Apple in the list of supported platforms for PyTorch. Nothing personal, I'm interested in the results, not the process.

> Don't remember seeing Apple in the list of supported platforms for PyTorch

Correcting myself, it's there. Which makes Mac Studio an interesting option.

Reply on news.ycombinator.com