Hacker News Reader

Comment by TobiasJacob | Hacker News Reader

I'm still hoping some future LLM is good and small enough for people to run local AI for everyday life.

It's interesting trying to be an AI "moderate," which as the owner of a 16gb card I definitely am. Still working on the balance, but it really does feel like I'm going to land on something like "use external AI services to build a few tools, but use my local rig to actually handle everything personal once those tools are built."

Already here. 3.8-27b is good enough.

5 years ago it would have been science fiction

Do you find it intelligent enough? What hardware are you using?

RTX 5090, AMD R9700, and MacBook M5 Pro (48GB)

For me, qwen 3.8 27b on my m5 max is so slow and dumb that I can't think of a single use for it. Haiku, Luna, or data center scale open models are ten times faster and ten times smarter (and inference so cheap that it's essentially free). It was a nifty toy to brag to friends about but why would I waste my time goofing with something so slow and dumb?

Try DeepSeek 4.1 Flash 1017, feels much better: https://github.com/drawthingsai/local-code-community

For roughly the same reason you might slow down and read a book on how to do something as opposed to speedrunning 20 youtube videos?

Are you insinuating that your agents are more effective at 20 tok/s than 200 tok/s?

I don't even understand your analogy. I'm not watching the youtube video or reading the book. The model is. And a big computer can watch 20 youtube videos faster than a small computer can watch 1, and do a far better job understanding all of them (bigger model at data center scale is both smarter and faster than smaller model at local scale).

On my Mac Mini M4 Pro with 48G I find Qwen 3.8 27B perfectly usable, useful and fast enough.

Are you trying to one shot create the Linux kernel?

Reply on news.ycombinator.com