You can of course "run" this on cheaper hardware, but the speeds will not be suitable for actual use (i.e. minutes for a simple prompt, tens of minutes for high context sessions per turn).
Not a "I actually use this"
The difference between waiting 20 minutes to answer the prompt '1+1='
and actually using it for something useful is massive here. I wonder where this idea of running AI on CPU comes from. Was it Apple astroturfing? Was it Apple fanboys? I don't see people wasting time on non-Apple CPUs. (Although, I did do this for a 7B model)
Language is full of issues of particulars vs universals, and you could debate if its just an integrated GPU with different marketing.
Whatever the case, we couldn't use it in production, and NVIDIAs stock price reflects the reality on the ground.
My 2 cents
Anyway, we were trying to run a 70B model on a macbook(can't remember which M model) at a fortune 20 company, it never became practical. We were trying to compare strings of character length ~200. It was like 400-ish characters plus a pre-prompt.
I can't imagine this being reasonable on a 1T model, let alone the 400B models of deepseek and LLAMA.
EDIT: oh its a marketing account and the program never finished... who knows the validity.
The tooling involved has improved significantly over the past year.
That said, I wouldn't necessarily recommend spending $20,000 on a pair of Mac Studios to run models like this. The performance won't be nearly as good as the server-class GPU hardware that hosted models run on.
e.g. in an office or coworking space
800-1000 gb ram perhaps?