I want to buy a Mac soon, and I'm really struggling to decide how much RAM I should order. Unfortunately, my budget is limited. If it wasn't, I would probably go for at least 32GB. I'm still hoping Apple might change their RAM pricing, but probably in vain.
I'd recommend getting at least 32GB if you're on the fence. Not being able to upgrade it is a bummer, and your future self will thank you for getting the most you possibly can.
For my most recent upgrade I went for 64GB (previously 32GB) and I'm really glad I did, especially since llama.cpp became a thing shortly after getting it.
Now I wish I’d bought 4tb rather than 2tb hard drive lol but that’s just me being lazy - that upgrade definitely felt like a step too far.
Here's Mixtral-8x7B-Instruct: https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-G...
64GB of memory will struggle on 65-70B (or bigger) models, and you'll be limited to running 70B on Q3 or Q4 if you want to use it somewhat comfortably.