theLiminator

Has anyone actually gotten good results out of this?

For the evals I've run (albeit, not super scientific), these memory systems did worse than the raw harness given ripgrep.

Yes for most devices people care to jailbreak though.

Not exactly imo, you could spend a crapton of money with white box pen testing with a bunch of frontier llms, and it might become a bunch harder to break into (it generally costs more to attack than it does to defend). Not to mention formal verification might become possible for more devices which might make it nearly impossible to find exploits.

Yeah, I've seen a lot of technology killed by acquisitions...

For example, https://github.com/Mooncake-Labs/pg_mooncake

I wish someone ran some sort of representative benchmark suite every X days to see if this occurs.

Perhaps coq/agda/idris/etc.

Imo, formal methods like more expressive/stricter type systems are key to making LLM generated code successful. Of course models will get better, but trusting the output will become much easier with a type system that proves more properties.

Yeah, local LLMs are probably an order of magnitude less efficient at a fixed level of "intelligence" if not more.

That's probably going to lead us to trusted attestation age verification.

Looks pretty cool, what steps are you taking to ensure that the backend is secure?

DataFusion is really cool, it's kind of like the LLVM of the OLAP world.

It depends on your quality bar. At a fixed level of quality, given a high reasoning sonnet vs a low reasoning opus, the low reasoning opus tends to be pareto optimal.

It's only when you need even lower levels of cost than opus at zero to low reasoning when sonnet starts to make sense at all.

This is useful research, but this particular model itself is likely absolutely useless.

What makes something `True AI`?

What's your test for it?

Please define AGI first.

Maybe something like Hylo? But personally I don't see anything displacing rust for the next few years, as I think there's enough rust in the training data for it to be the best "serious" language for agentic systems-level development.

It's really the only systems language in its exact niche.

That's the power of a strong test suite. LLMs excel when you have verifiable rewards. I imagine we'll get a lot more rewritten in rust projects in the future. Rust is also an ideal target for such rewrites as it offers a lot of verification (via its type system) and is low overhead with zero-gc. There's less and less reason to use GC'd languages in the agentic coding era.

I think Rust is a locally optimal target for LLM coding, we might see a better language in the future, but I think Rust will dominate for quite some time.

> Much more than any other country on Earth.

What about Canada?

If anything I think discipline and rigor will go up.

I think it will force us to adopt stronger type systems, formal methods, and more automated verification.

Seems like the way to go for any smaller models is to only use the low reasoning levels, and for anything where you'd want it to reason harder, to just use a larger model.

In effect, high reasoning only makes sense when you're using the frontier model and need extra performance (higher levels of reasoning are never pareto optimal unless you're at the largest model size).

This is the type of problem for which LLM generation is great for.

If you have an oracle, and your problem is largely just a pure function, it's pretty good at generating something that both works and is fast.

How does this compare with vega/vega-lite?

I personally skip breakfast and just eat lunch and dinner.

I'm not very active, and I've found that doing that as well as not eating snacks, sugar, or having calories in drinks makes it pretty easy to roughly be calorically neutral day to day.

This take is ridiculous, the PRC is not going to care at all about US regulations.

I don't disbelieve a 5000x speedup is possible, I disbelieve that a modern day supercomputer will fit in your pocket in even the next 10 years.

Yes, I'm talking about a supercomputer from today in your pocket. That probably requires at least 5000x perf/watt if not even more.

> But, history says the supercomputer of today will fit in your pocket in a few years.

I don't think this will be true in the same time span anymore. Each miniaturization is costing more and more money.

Perhaps they'll come up with exotic fundamental improvements, but I don't think the rate of improvement of compute/watt will match the previous decades.

> it's a cat and mouse game that favors the defenders IMO

How so? I'm actually against most of the "safety-tuning" that anthropic does, but this seems fundamentally untrue, a close analogue being video game cheat development. I think in general the cheat developer has an advantage and the cheats generally proliferate for quite a while before being patched.

This is pretty bullshit, now you have no idea if your output is getting silently nerfed.

> We have also added safeguards related to frontier LLM development. As discussed in Section 6.1 of our February 2026 Risk Report, we are concerned about the risks of accelerating the overall pace of AI development, though we remain uncertain about the severity of these risks. In particular, our concern is with—as we wrote then—“accelerating other AI developers in building powerful AI systems that pose similar risks to the ones ours pose - without necessarily having commensurate safeguards.” In light of the ability of recent models to accelerate their own development, we’ve implemented new interventions that limit Claude’s effectiveness for requests targeting frontier LLM development (for example, on building pretraining pipelines, distributed training infrastructure, or ML accelerator design). Using Claude to develop competing models already violates our Terms of Service, but enforcing this restriction through our safeguards avoids accelerating the actors most willing to violate these terms. Unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user. Fable 5 will not fall back to a different model. Instead, the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning (PEFT). These interventions will not affect the vast majority of coding work. We estimate they will impact ~0.03% of traffic, concentrated in fewer than 0.1% of organizations. When these interventions are active, we expect them to have minimal behavioral impact on the model except to limit its effectiveness in developing frontier LLMs. Claude will still respond helpfully to user requests. We’ll continue to improve the precision of our detection methods following the launch of this model.

This seems pretty bullshit, you're paying through the nose for tokens and if you are doing anything ML-adjacent, you might silently get worse output without knowing it.