Silagi

There was some (admittedly flimsy) evidence from a microsoft model description card that it's doing multiple passes through its own weights before token generation to improve output; it might be an architecture change that's making it slow, moreso than raw size

Part of it is putting them in the right world and harness; I've been working on implementing bots driven by models in a 20 year old MMO with a relatively simple interface, and just a few weeks into developing the bot/model features and contracts, I'm already seeing glimpses of interesting emergent behavior in model driven worlds; alliances form, hostile players get exiled, and teams try to support each other.

From here, it's a matter of dialing in what's "the bot" vs what's "the model", then working through how to integrate them with human players. Obviously I don't have any guarantees, but it does seem like some interesting gameplay could come from it. Having a real sandbox style world with multiple clearly quantifiable goals that are at odds with each other seems to be a key way to push the models towards interesting, emergent behavior.

Also, if you haven't seen it, check out the Emergent Gardens youtube channel; it's one of my favorites, and they've been exploring these ideas for a while.

That seems like the least charitable reading of what I wrote.

"system they're working on" refers to the actual physical hardware.

Yes, I look at the code regularly, and a significant portion is hand written, especially the performance critical sections.

A security issue I haven't been able to track came up in automated code reviews. Maybe you're such a savant that you can crack any security issue in a codebase you're working on, but the last few months of critical security bugs in infrastructure with the most experienced and capable maintainers in the world should at least humble you a little.

That was my experience in the Opus 4.5 - 4.7 era, but recently it's been on the nose with what I want it to carry over, or close enough that a full handoff would be more expensive than just letting it find what it needs again.

I'm mostly working on moderately sized rust codebases (600k~ lines) with reasonably long running tasks. The only time I intentionally do a handoff is if the next goal is in an entirely different section of the code, or I let the session cache expire, or I'm working with 6.1 Sol for more than a few hours, which admittedly has some horrendous context drift compared to Opus.

Can I ask why? The modern models deal with compaction remarkably well, to the point where I dropped my autocompact from 1M to 350k. I've seen no change in efficacy and a reduced usage rate.

I've been working on a game server in rust, and every once in a while a review will just stop and refuse to explain what happened. So I know there's a security issue in there somewhere, but neither OAI nor Ant models will tell me what it is. Despite the whole thing, top to bottom, having been built in the last two months on the system they're working on.

Planning to hop on openrouter this weekend to chase it down, but man, it's rough knowing I've already burned the tokens to find it probably half a dozen times at this point, but don't have any information that would help point me at it.

A lot of the work is carried over model to model, so I think the most comparable product is a live service game. Always have to be building on the last thing, and if you stop you'll essentially lose the entirety of the initial investment because you won't keep anyone's attention. Plus a healthy dose of "winner-takes-most".

Just. Yaknow. 100x the costs, at the least.

Maybe it's my tasks, but in the cases you're describing, I'm usually better off going down to Luna. I specifically don't see the gap where "It's too complex for Luna-high" and "It's not complex enough for Sol-low" exists.

Can anyone give me the compelling reason for Terra and Sonnet to exist?

Virtually every task I give to either is better served by going up a tier and down a reasoning level, or down a tier and up a reasoning level. I simply don't understand their use case. No testing I've done has shown them to have a niche where there isn't a cheaper-with-equal-results or more-performant-at-same-cost option.

That's wild to say when comparing to the OpenAI family of models. I've never seen so many tests that validate tests that validate tests.

Not OP, but it launched in June 2022, so less life than I'd like to see from a Thinkpad regardless of how you slice it.

The early image models had a sort of urban legend briefly developing around them about this mishappen, horrifying woman that would appear if you just let the models recursively recreate the same image over and over.

There was a weirdness going on with the first Midjourney models that I miss.

The videogame industry, in a nutshell.

Further, this benchmark looks to have done too poor a job of canvasing the model/reasoning level space to claim to represent the pareto frontier. Only running 6.1 sol and opus 5.5 on max seems absurd given that their other reasoning levels represent almost the entirety of the pareto frontier above luna for coding.

Is it low quality in the "It didn't match the meaning that was conveyed" sense, or in the "It didn't capture the exact words used" sense?

My reaction to seeing the project was that being an embedding model it'd be pretty good at conveying the meaning but not great as using the same terms, is that what you're seeing?

My experience has been that most people don't even understand what computers were capable of before LLMs; I work in a chemical company, and I can count on one hand the number of people who knew SAP has GUI automation built in.

Don't even try to explain that "copy data from X program to Y program" is something computers should have been handling for the past 20 years. It's been someone's job the whole time.

We haven't cleared the hurdle of "Processes a 100 line python script could automate in 2010", I don't think most people will be leveraging LLMs to their potential by any stretch.

Every test I've done with Sonnet or Tera has been more expensive than just using their big brothers, even before 6.0 and 6.1 Sol came out at Tera prices.

They're cheaper per token, but they burn more tokens and take more wall time than just going up a tier at a lower reasoning level. And if the task is simple enough that they do come out cheaper than Sol, I can switch to Luna and come in even cheaper.

Yes, but they're still going to encounter those moments, just at different points.

The same way we never stopped encountering "Wait, what the hell?" moments when we had the internet and could search for problems instead of digging through textbooks or --help, they'll just run into those blocking moments at a higher level of abstraction, then have to backtrace the problem down to the level they need to understand to resolve the issue. Curious people continue to be rewarded with a higher ceiling, but the floor to get something functional is brought down.

The alternative is that AI just deletes software engineering as a discipline and everything gets vibecoded, which still seems unlikely even if the improvements continue to accelerate.

It's far more likely they start to restrict the usage of subscriptions in corporate settings and leave the individual subscriptions to inflate the margin against them away. The individual subscriptions are the hook; people need to be able to understand what they're capable of with enough tokens to tackle real projects they couldn't have without the sub. That's the only way individuals will make the attempt to sell it to their org.

We'll probably still be getting the same amount of work done with a $200 subscription a year from now. That will just represent a much smaller subsidization than we currently enjoy; something like 3:1 - 6:1 instead of 40:1. Maybe running at a lower tps than the API gets. Maybe no access to the absolute frontier, but still significantly more intelligent than we get now. The labs will essentially break even on the subs and the corporate spending will be the profit center. Tale as old as software.

For simple tasks that require a lot of input tokens (e.g, "Figure out the full path this function calls through and give me a map", or "Summarize these 5 PDFs and give me the main ideas I should explore") you can fire off a Sonnet agent at low-mid reasoning at it'll be cheaper than if you had Opus do that summarization itself.

Now imagine you have a set of twenty of those tasks. You launch an Opus agent, give it the task list, and tell it not to do the work itself, but orchestrate agents to perform all of the tasks and do small spot checks to verify the work.

The overall task is completed much faster at a similar or cheaper cost with an extra verification layer inserted that wouldn't have been there if you just used Opus.

See also: Cyberpunk's Blackwall

Key issue driving any conversation about desktop environments; so many people have vastly different workflows and expectations for their computer.

Personally, I keep my desktop absolutely sterile, usually some geometric patterned background, and that's it. Autohiding taskbar, no shortcuts, etc. It keeps me from having the "stepping into another room" effect when I sit down at my computer, which I'm very prone to. That means I can hit win/option/spotlight depending on the comp and launch the program I sat down to get into, without thinking "Oh yeah, I meant to edit that excel file..." and going on a 30 minute tangent.

I felt a little panic when I understood every word of what was meant to be adjacent to scifi-style garbled engineering speak, and I think I'd rather be on your side of the spectrum.

Realistically, it's never going to be Pi, for the same reason that video game consoles are always more performant than PCs for a given hardware spec.

There's no canonical Pi harness, it's too variable. There might be a world where something Pi based is the target, like OMP, but then you have to hesitate when you start extending the harness because you don't know what effect any given extension will have on model performance.

The irony is, this would be a pretty trivial one shot project for a modern LLM to create that interfaces with the source control repository of your choice. It's basically an advanced note-taking app, which LLMs have been trained to death on. I bet you could get it to write it as a VS Code extension as well (or emacs macro, or... etc.)

I've been running 2 threads of Opus 5.5 for ~16 hours on a server C++ to Rust translation/optimization project and used 8% of the 20x sub. Going to test out gpt6 sol over the weekend, but it really seems like we're back in the realm of having to try to burn a 20x sub.

I think this is where Deepseek has nailed the mark; DS4.1 Flash is really, really fast, and really, really cheap. If you give it small, structured goals, it completes them crazy quick, at negligible cost. There's different vectors to differentiate along to stay in the conversation.

I've taken to using them as micro-review subagents at development milestones, where a "frontier - 1" model like Opus or Sol launches 10-15 of them on small review tasks that each run for ~10 minutes. Costs about $1 per cycle, and they usually catch something Astra or Fable didn't. Then the orchestrator validates each claim before passing it back to the planning session so we can fold the findings in.

Adjacent, I work in a lab so we have dozens of ovens and freezers running constantly, so I'm intimately familiar with the constant hum, even though my office is a couple of doors away from the oven room.

We had a full site shutdown last week, which involved all power being shut off, which meant no running ovens.

The lack of sound felt so strange, like I was somewhere I shouldn't be, or in some sort of liminal space. It was unsettling in the silliest way.

This is really the only reason I have a MacOS system around. The M4 Air feels like what I imagined a laptop could be 15 years ago. No fans to block, can grab it and carry it around almost like a tablet, works great as a thin client into a more powerful system over Moonlight, it's light enough that I don't even think about carrying it around in my backpack even when I'm also carrying my work laptop, and it was cheap enough that if anything happens to it I won't be too concerned.

Compared to the last XPS I had, it's substantially better for my use case, since I'll generally be remoting into systems anyway.

You could realistically be burning through ~5k of tokens without hitting the quota. The subscriptions are crazy subsidized relative to the API rates.