- one one hand, the 8GB RAM Mac mini with M6 as cheap as it maybe is an absolute waste of money when it comes to running LLMs
- on the other hand the most expensive machine is right here
- what is the cheapest most expensive machine you can buy capable of providing you the best ROI for running LLMs locally?
But if your use-cases need extremely heavy-duty local models then just open your bank vaults and get the one you need.
For just trying out local models and running occasional queries a 16GB mac will run an 8B-20B dense quantized model just fine.
If you are running at the scale of OpenAI then a data center full of blackwells will have the best ROI.
take an 8 year life cycle for all machines and think of the cheapest expensive machine that you can buy which ll give you a decent llm to run
To do significantly better than that with a Studio, say M5 Max with 128GB would cost twice as much again. It would probably be more than twice as performant and much more capable, but you'd be getting into very big bucks that IMHO would only be justifiable for paying pro work.