skohan

I found Glimmer underwhelming in terms of coding - I tried it as a drop-in replacement for 3.6, and the output was noticeably worse. 3.8 has been a significant step up so far from early testing.

It's only been a couple days, but I haven't seen looping issues with 3.8 so far, compared to 3.6 which did occasionally have this problem.

At that point aren't you just edge-case testing?

Surely most of your use-cases are not novel tasks that combine obscure domains.

It seems to me the real way to evaluate the value of a model is how it performs in your real-life workflows.

Depends on your use-case. Over the past couple days, I've found 192k context more than enough for coding. There's more thinking for sure compared to comparably sized models (running on xhigh), but I've found the results are so much better that the entire session consumes less tokens on average since weaker models need more review passes.

Imo it makes sense for things to move in the direction of small, focused models that excel in one area. I use LLMs for technical work 99% of the time, I could care less about general world knowledge, or if the model is good at creative writing.

With good orchestration and delegation you can get surprisingly far with small models running on consumer hardware.

I'm running 3.8 27B locally, and the results from the past few days have been excellent. I find raw speed is less of an issue when you can trust the model more to reach the right result.

That could be a move towards regulatory capture. Enact standards that they (and OpenAI) largely control, block access to Chinese models in the US, and effectively prevent challengers from catching up.

At the same time they slow down the arms race, so they can back off on training Capex without losing their lead.

Isn't this meta release kind of a counterexample to that?

There are exciting developments underway in analog computing.

The power of small models isn't only that you can run them on local hardware. You can also fully own your data and workflow, and choose/fine-tune models for your specific use-case.

for clarity, I'm not agreeing with GP that small models will mean doom for data center projects

I'm using Qwen3.6 27B Q4, max context with pi on 32GB VRAM (although I'm testing out Glimmer on a feature implementation literally right now). Pi is great because it has minimal context added by the agent.

Looking forward to the 3.8 27B release to compare.

> write code to do this in this file

I haven't had to micromanage to this level. I usually start with a spec for a feature, which will be as detailed as I am opinionated about the feature. But it's usually on the level of a high-level context, plus some key implementation details (technology choices, key requirements, maybe an interface/API specification to 80% detail), and then the project already has high-level policies documented about e.g. how to structure files within the project.

Then I do a planning phase, task breakdown, and implementation of subtasks all within the model. I do read through it, but mostly the quality is good and I might make a couple notes. Then I do a review phase, which usually picks up a couple things. I'm moving towards less manual review of results and more automation as I learn what I can and can't trust the model with.

There's definitely a capability gap vs. larger models, but honestly I kind of prefer this workflow, as I stay more in touch with how the codebase is structured.

And it's great to be able to experiment as much as I want without worrying about how many tokens I'm burning or how close I am to a usage limit.

Yes they have quants for 32GB and 20GB use-cases (including mmproj and kv cache + context)

It should - the kquant-dynamic variant is targeted towards 32GB. Downloading it now to give it a try.

If you don't mind exfiltrating all your IP to the API provider

Still great if they want to play in this space. Having competition for the 24-32GB VRAM target is only good for the end user.

I've been coding using the LLM server in my living room for the past few weeks, and I haven't had this much fun with tech for ages

I could imagine pulling out all the stops to get a release over the finish line a week early if you're worried about being surpassed by another release

5 years ago maybe. That juice has been squeezed.

The way this reads, it seems like it was written for US lawmakers.

> the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.

This comes off a bit hypocritical, coming from one of the men responsible for the models most likely to be used by the US military, and for repression of US citizens.

> We should crack down on industrial-scale distillation operations ... We should have policy interventions to deter this behavior.

How exactly would you design policy interventions to stop distillation? If Anthropic wants to make their products available around the world, you would need some kind of global regulation to curtail it. And you would need to find some way to enforce it, which is far from trivial.

Honestly I don't see how securing models against distillation is up to anyone but Anthropic, if that's something they want to achieve. Calling for regulation is a bit like crying to mommy and daddy when things aren't going as you would like.

--

Overall, I can understand the argument that advanced AI models can present risks. But I don't see how regulation within the US can mitigate any of those risks. Any bad actor would be able to access the models outside the US, or covertly access the weights.

The only real way to prevent advanced models from being deployed would be to go around the world bombing every significantly powerful data center, and I don't think Dario would call for that any time soon.

The only thing this kind of regulation would achieve would be cutting US companies out of half of the innovation happening in the AI space, putting the US at a disadvantage relative to the rest of the world at everything except maybe frontier AI development.

Yes 18 months ago it seemed like AGI was being promised every other week, and now I don't see any of those headlines.

The problem some have pointed out is that these companies are such a huge portion of the market right now.

The sound advice for the past decades has been, just invest in a low-cost ETF tracking the S&P instead of picking stocks to minimize risk and invest in the market broadly.

So a huge number of people have done that, believing they're diversified, while tech makes up 40% of the index.

Yes you could sell your S&P and find things to invest in least likely to be impacted by a potential bubble, but your average 9-5'er with automated contributions to their 401k is probably not sophisticated enough to do that.

And that's assuming only these companies would be affected if there was a massive draw-down in tech/AI related stocks. We haven't really seen a situation like this before, so it's not easy to predict what effects there might be in the broader economy.

Couldn't it be a problem given the concentration of the S&P in these companies?

At this point these companies make up a huge portion of 401k's for a huge chunk of Americans. How would it affect retirees if they dropped 40-50%, likely taking the market with them?

If it didn't matter, why would they bother jumping through hoops to keep the debt off their balance sheet?

In the run-up to 2008 a big factor in the bubble forming was that poor quality loans were packaged in a way to hide the risk in those investments. I'm not expert enough in finance to know if it's the case now, but we do know that clever accounting to hide debt can lead to the incorrect valuation of assets, potentially leading to financial ruin.

I wonder to what extent the 4o rollback was motivated by this exact case

I had the same experience. For me peanut butter and apples were my "healthy snack" and were accounting for a huge amount of calories per week.

Not only forests but even just tree cover. I live in Berlin and there were a few days a few years ago over 40 degrees, and I remember the difference was stark going from my relatively tree-lined street to an area of the city with only concrete.

I always felt unwrap is preferable for diagnosing issues, which makes it useful in tests. A failed unwrap will point you directly to the line of code which panics, which is much simpler than trying to trace an issue through many layers of Results.

If you use `assert` in tests, I don't understand why you wouldn't also prefer unwrap in this context.

I think it's also perfectly reasonable to use in a context like binary you run locally for private consumption, or for instance a serverless function which allows you to pinpoint the source of errors more easily in the logs.

It's not a good fit for cases where you do expect cases where the unwrap will fail, but it's a great fit for cases where you believe the unwrap should always succeed, as it allows you to pinpoint the cases where it fails.

I was a huge Swift fan but SwiftUI and the changes supporting it in the language got me to switch to Rust for all my personal projects

Doesn’t rust’s static linking also have to do with the strategy of aggressive minimization? Iirc for instance every concrete instantiation of a dynamic type will have its own compiled binary, so it would basically be impossible for a dynamic library to do this since it wouldn’t know how it would be used, at least not without some major limitations or performance tradeoffs