Hacker News Reader

Comment by SparkyMcUnicorn | Hacker News Reader

> I want to buy a Mac soon, and I'm really struggling to decide how much RAM I should order.

I'd recommend getting at least 32GB if you're on the fence. Not being able to upgrade it is a bummer, and your future self will thank you for getting the most you possibly can.

For my most recent upgrade I went for 64GB (previously 32GB) and I'm really glad I did, especially since llama.cpp became a thing shortly after getting it.

View on HN

Also in the “glad I got 64gb” camp - even though it seemed ridiculous when I bought it, technology has advanced so quickly that now it’s actually very useful.

Now I wish I’d bought 4tb rather than 2tb hard drive lol but that’s just me being lazy - that upgrade definitely felt like a step too far.

Would you recommend the 64 over 128? What kind of models will 128 open up over 64? Can you get most of them out of the 64?

For LLM models, TheBloke has provided memory requirements for all of their quantisations. For Apple Silicon you want to look at the GGUF models.

Here's Mixtral-8x7B-Instruct: https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-G...

64GB of memory will struggle on 65-70B (or bigger) models, and you'll be limited to running 70B on Q3 or Q4 if you want to use it somewhat comfortably.

I’m not sure, not really an expert in this field, just casually interested as an outsider. When I bought my Mac, 64 was as high as you could go.

Reply on news.ycombinator.com