dofm

Alas Opencode's source is hideous to read.

When you say "kinda dumb" it's worth noting that the smaller coding models tend not to actually need system prompts to be helpful coding assistants, etc. Qwen maybe more than Muse Glimmer? I've not tested Qwen 3.8 27B enough yet. Even Gemma 4 didn't seem to need much.

Muse Glimmer is so absolutely tuned for one particular pattern (turn-by-turn agentic coding and assistance) that it will do it more or less without a prompt at all (except to set the reasoning level maybe). Beyond the tools announcement, it feels to me like it almost only needs a prompt to do plan mode stuff. If LM Studio (in either of its forms) or Unsloth Desktop had proper support for running inside a linux container rather than just a filesystem sandbox, you'd barely need a dedicated coding agent at all. Osaurus gets even closer.

I wanted to like Opencode but I can't, really. I am enjoying Pi slightly more because it's just way less convoluted and obscure.

FWIW I use Paseo for the GUI, which to some extent eliminates that aspect of decision fatigue; if you like Paseo's UI well enough, you can have it with almost any harness. And since it can connect to its daemon via ssh you can still have the desktop GUI with a container. (Not clear to me if Opencode v2 Desktop makes remote connections to the daemon or just wraps the web interface like v1 did)

The point is that the level of imprecision in swipe means there's more guessing; they don't want to guess things that frustrate the user. It is not censoriousness, it is minimising surprise.

> And those fruits haven't changed or improved noticeably in decades.

Strongly recommend you get Gemma 4 12B QAT on a computer that can run it, and then disconnect the internet and see what it can still do. Even the smaller E4B will show you.

Local AI is the first thing in decades (since editing video, perhaps) that meaningfully changes what a disconnected machine is capable of.

My phone use dwindled to almost nothing after my father died. I unplugged my landline then because he was the only real person who dialled it.

I very rarely get calls on my mobile; only about one in twenty aren’t scams.

Right. And FWIW those people are actually quite close to being able to do what they need with their phones.

If it were not for the x86 architecture they would almost have models that could answer those queries on their desks. (Gemma 4 E4B QAT is surprisingly good at this stuff; Gemma 4 12B QAT is great at it).

If I was Sam Altman I would consider giving Intel and AMD money to not develop unified memory architectures.

You can sell your DGX Spark for basically no loss. Maybe even make money.

> Mines based on libvirt and uses cloud-init for creating the images, with shared golden images across projects.

This sounds good. Cloud-init is still worth the time, I think (as I just said in another comment)

"You know you should run your code and agents in a sandbox, but most tools need configuration and setup that’s hard to get right."

This is very true. Having been a user of heavily-customised vagrant boxes for a long time I've been spending a fair bit of time lately on updating my skills for this.

But my own stupid old-fashioned view is that it's probably worth learning cloud-init if you never have. Deploy your VMs on your own machine with multipass or whatever, and the same deployment works with little change in the cloud.

That's been the approach I have settled on, but only after a disappointing experience with Apple's "Container Machine" , which is both quite bothersomely insecure by default (your whole home directory? mounted rw? in a VM with an agent? really?) and from there sort of cumbersome/idiosyncratic.

Docker SBX is neat but the Docker account login aspect is annoying to me.

There is an SBX-like example in the Apple Container examples repo:

https://github.com/apple/container-examples/tree/main/contai...

Keep meaning to try it.

"Not bad - scale is harder than it looks."

Coincidentally, so is good web design.

One thing that really really annoys me about LLM-generated websites is that they have entrenched, without good reason, --webkit-font-smoothing: antialiased.

Also Claude, having evidently been tested by people exclusively under the age of 25, routinely chooses smaller font sizes and lower text contrast than makes any kind of sense.

I know a couple of people who have eminently LLM-capable GPUs more or less doing nothing when they are at work…

3D games give me motion sickness so I don't have powerful PCs anymore. In fact right now I don't think I have a desktop/server PC that can power up at all, and that alone makes me wonder about building an R9700 machine.

Remember they literally had, and cancelled, Steve Jobs's real-postcard-from your-photos passion project.

Better yet, prompt ChatGPT to do it.

They really went to a lot of trouble to do this when none of us answer the phone to unknown numbers anymore.

Right. Nobody says it's necessarily economically the best option. But it's also an asset that has alternative applications and barely depreciates!

Right now you could extensively use an M5 Max for a year and sell it for almost exactly the same amount at the end. Maybe even more, at the current rate. And it can do other things for you: host your VMs, edit high def video, whatever. Even if it stops being useful to serve LLMs, it might be useful to run as a server for other tasks for a whole decade. It will be a fast, supported Mac for seven years. For the R9700, maybe gaming?

Depends what you want, doesn't it? Or are we all expected to conform?

For me personally, right now I have a secondhand M1 Max macbook pro, and that is my local AI machine, because my needs are exploratory and I want to run it here. I want to learn by comparing models. I want to learn how to run them and advise others how to (local inference is not a zero-sized market).

My next step up will be to take the apps I'm looking at building that run against these small models, and host those apps online with some sort of cloud inference — openrouter, digitalocean, cloudflare — for those same models.

If I ever find I want to vaguely describe some desktop application before bed and come down the next morning to an aggravating slop version of it written in Rust, I'll consider Claude.

For a local non-laptop, I've looked at the M5 Max and the R9700. I've also looked at the M5 Pro, which in practice is about as fast at token generation as my M1 Max (because the M1 Max's memory bandwidth is strictly theoretical), and smokes it at prefill. That alone might be enough for a "leave it running" mini AI box, and it will be a nice desktop machine. I just don't need it enough yet.

As an aside there's this thing — somewhat interesting, though constrained:

https://www.olares.com/one

Uses the 5090 mobile chip, which is pretty good speedwise (about the same as the R9700) but 24GB. Small, tidy, on-prem. I think they have missed the boat on pricing, but if they can ship in volume, maybe not.

> If you actually require intelligence to get a hard job done, you should just use a frontier model.

Or, you know, your own intelligence.

What I do not understand as an engineer is how our community has gone from believing in open source for controlling destiny, security, etc., from avoiding unnecessary corporate capture (think of all the rants about Swift!), from believing in heterogeneity and portability, to just saying, look, if you're not using the product of one of two loss-making AI unicorns to do all of your work, you're just being a contrarian weirdo who is doing it for performative reasons.

I guess if that is your definition of contrarian — that everyone else is doing it so why aren't we? — then I am.

But in my experience it's not so different to "why aren't you on social media?" — which was an argument from insecurity and the desperate need to fit in.

I personally believe in independent thinking. I have things I want to learn about how all this actually works, I have questions about my own life and future, and I'm not going to simply chuck them away because you or anyone else thinks it's just being contrarian and we shouldn't question the direction everyone is taking. So I am learning the tech with a stack I control, while continuing to do the job I do.

You can do what you want. You're probably right, and I'll be quite happily wrong and continue in my wrongness even though, based on how you characterise it, this kind of independent thinking appears to unsettle you.

The RTX 5090 is the ideal, but it's not the only 32GB GPU that can offer workable performance; the AMD R9700 is catching up.

You can surely build an LLM server with an R9700 and 32GB of mainboard RAM for less than the 64GB M5 Max Apple Studio, which is probably the next most competitive model up. It'll be better than a Strix Halo but less flexible.

I'm not offended. I'm also not the person resorting to insults.

Ref: https://lists.gnu.org/archive/html/emacs-devel/2015-02/msg00...

No don't worry. It's just allergies.

Putting aside that you're "really strongly" agreeing with a personally directed insult:

You should factor in that undeniably most people don't have the means to run even a 27B dense model locally, and therefore do not actually know what they are and are not capable of in that context.

And you might want to look at openrouter's rankings.

If you are comfortable automatically assuming that "doesn't use Claude or Codex" is equivalent to "not a good engineer" or "bad engineer", then I think you are probably lost to more nuanced discussions.

> It has no experience with being disliked by normal people.

I mean, crypto and NFTs ought to have given them a taste of it. But then perhaps it did, and they are now inured to it, because it's a lot of the same guys, including Sam Altman.

> You see this constantly, but as usual this is management refusing to put in the least amount of effort.

We're all managers now. So everyone can be that lazy.

Ahh yes. Those are the only two options.

Well it's certainly the service-dependent (addicted!) model that the big two want.

But will you want to do that when your tokens are metered at non-subsidised prices, and yet there is absolutely no clarity (ever!) about token efficiency, how the model decides to spend your money, how it might ever decide to stop wasting your money and get to the point, etc.?

The specify-loops-and-walk-away model absolutely points to local/on-premises/private cloud AI.

Because otherwise you're letting Anthropic or OpenAI decide how to spend your money. And they are going to be biased towards spending more of it.

I don't think Qwen 3.8 is quite there yet with the loops stuff. Definitely seems to be better at deeper and deeper one-shot puzzles.

A few more will have Macs with 48GB RAM though. (Indeed if you push it a little, 36GB unified is possibly enough for Muse Glimmer; I have not tested this theory)

There is, for sure, a limiting factor in the x86 architecture, but I think the RTX Spark will change that a bit, and maybe AMD's subsequent Ryzen AI/Strix Halo chips.

Would you talk to me a mo about Kilo compared to Pi?

I am digging in on Pi (in lovely Paseo and exploring Pendant when in VS Code).

What would I find better or worse in Kilo?

> The people saying that local models can easily replace the real Frontier Models from Anthropic or OpenAI are just not using it for the same things you and I do.

> They seem to be happy with their small usages.

Oh yes, we're terribly inferior people. Just living on scraps from the masters' tables.

As amazing? I suppose not.

Good enough? Quite probably, unless you're in the "coding is solved, specify loops and walk away" frame of mind.

32GB (or say a 48GB-unified RAM unified machine) gets you Qwen 3.8 27B at a good adaptive Q4, maybe Q5. Muse Glimmer at Q4 and Q5, etc.

These models can code well enough in an agentic way, I find, and the chat-to-assistant scenario is just fine in Unsloth Desktop or LM Studio.

A slightly lighter harness helps; a lighter system prompt is entirely possible, especially for Muse Glimmer, which is post-trained so effectively for agentic helper tasks that it barely seems to need a system prompt at all except for a "plan" mode.

Tool use is all fine, all that.

And you can do that without even being near the vehicle, because the panels fall off.

Tired: all terrain vehicles

Wired: special purpose vehicles