Hacker News Reader

Comment by paxy | Hacker News Reader

Can't really answer that without knowing your use case. What is the largest model you need to run? What are you going to do with it? What tps do you need?

For just trying out local models and running occasional queries a 16GB mac will run an 8B-20B dense quantized model just fine.

If you are running at the scale of OpenAI then a data center full of blackwells will have the best ROI.

View on HN

the use case is getting the best bang for your back per dollar

take an 8 year life cycle for all machines and think of the cheapest expensive machine that you can buy which ll give you a decent llm to run

Cheapest expensive? I'd say the M5 Pro Mini with 64GB RAM. That's certainly not cheap, but not ludicrous, and fairly capable. An M6 mini with 32GB would be the budget option and quite limited, you'd probably want to replace it after say 4 years.

To do significantly better than that with a Studio, say M5 Max with 128GB would cost twice as much again. It would probably be more than twice as performant and much more capable, but you'd be getting into very big bucks that IMHO would only be justifiable for paying pro work.

I recently bought the M5 Pro Mini with 64 GB of RAM, and it feels excellent. It matches the Max in CPU performance while running Qwen 3.8 27b. Although it is slow, I can complete tasks at what I would call Opus 4.6–4.8 levels. What surprises me is using Claude Code on it. The speed of building with that harness and testing iOS apps is impressive… full end-to-end control of Xcode through to TestFlight. This is a major upgrade from my old M1 Pro MBP.

Reply on news.ycombinator.com