Hacker News Reader

Apple M series chips are already some of the best chips for running your own models locally, especially if you want a laptop or small form factor machine instead of a big gaming desktop with a big GPU.

Apple is really getting a niche here for machines to run models locally. That’s pretty powerful.

View on HN

I would not say they are the best chips if you want to run locally. They are ok although a bit slow compared to desktop gpus for inference.

What desktop GPUs would you recommend for the home user with access to 20+ GB of VRAM? I can get a Mac Studio with 128GB of shared RAM for about 5k.

What's the point of shared RAM exactly? 2x3090 ($2000) + 128GB RAM ($300) to cache weights would be much faster. Probably around 10-20x.

> What desktop GPUs would you recommend for the home user with access to 20+ GB of VRAM?

I think the usual current model would be the 24GB Nvidia RTX 4090.

> Apple is really getting a niche here for machines to run models locally.

I... don't agree. They have an acceleration API and a large install-base, but all of these models have run just fine on traditional hardware. GPT-Neo, Stable Diffusion and LLaMa can all be used and accelerated without Apple Silicon.

Powerful, maybe. But not really unique, just putting up table stakes.

Reply on news.ycombinator.com