AFAIK the timeline was:
- AI was shit, models could fit in your GPU
- AI was shit, models could no longer fit in your GPU
- AI got good
Is $2k for a whole machine not "consumer" ?
llama-ui on unsloth/Llama-3.2 3B Instruct:
> hello
Runtime info
Current time is 2026-11-02T13:00:18+01:00
Runtime info
Current time is 2026-11-02T13:00:19+01:00
Runtime info
Current time is 2026-11-02T13:00:21+01:00
Runtime info
Current time is 2026-11-02T13:00:23+01:00
There is not enough information about the functions to properly answer the prompt.
I did do the yak-shaving for the longest time. llama in a second-hand RTX3070 8GB was a useful tool. When gpt-oss came out, I upgraded to a second-hand 16GB card. It was slower (didn't seem to one-shot as much) but it gave smarter overall answers. At some point I bought a 24GB card which was basically dead on arrival. I kept my 16GB card in a server and maintained my VPN infra so I could access it from wherever.
I was still using this setup even when my workplace was pushing windsurf/copilot harder and harder. For me they were just broken. I'd ask for trivial tasks like "copy this unit test into a second unit test which tests something slightly different" and it would just get confused and delete shit, or leave random UUID strings throughout the codebase.
Eventually I tried out cline/openrouter/deepseekv4flash, and that was actually useful enough for me to tell it to do something to my code base, and as long as I supervised carefully enough it was useful, and priced OK.
At some point in the process I tried Claude Code because of all the hype, but I couldn't get any useful value out of it. I tried working with it all morning on something and hit my token limit without achieving anything worthwhile, so I cancelled Claude and went back to Deepseekv4flash.
These days every job application requires proficiency with claude code (or similar), so I was forced to get it again. And this time it just worked better. I haven't hit any limits, and it works like a better version of my cline/deepseek setup.
So where does that leave me?
Deepseek-v4-pro is 1.6T params, the cut-down version is 284B params, and I wouldn't trust them to do something on their own.
I'm very skeptical of a model being both small enough to run fast on my hardware and somewhere in the v4-flash <-> claude quality range.
If I did put down $2k, I wouldn't want it in such an old chip (M1) or a second-hand device that could die tomorrow with no chance of refund, especially to just 'see if it's good enough' (compared to the effectively-free v4-flash token spend).