Hacker News Reader

Comment by nejch | Hacker News Reader

This is based on one of the smaller Qwen models, just like Cloudflare's Clef, Strands decider, and a plethora of others released in the last couple of weeks.

Kind of funny how much hype they can all get out of this, but Qwen really is the little engine that could. Great to see open weights (if not open source) driving the whole ecosystem like this though.

View on HN

My biggest learning after some experiments - a BF16 (unquantized) Qwen beats a Q8 of double its size for decisions. I guess that’s the reason Kev switched to 4B BF16, from the original 8B version. Isn’t it interesting that quantization seems to mess with decision accuracy?

That’s fascinating, but not that surprising to me. We act like quantisation is free “Q8 is basically lossless” is often said in the local LLM community, but it really isn’t. The trade offs are worth it, personally, and the damage to coding ability seems low: decision model approaches are stricter though

Super cool finding!

Interesting - I wonder if it's because coding doesn't use the specific token probabilities, while decision models do

I think it also helps that speed and latency aren't as vital for coding and we can afford to let the model think for longer.

Yes this is the reason. Transformations like quantisation preserve the rank of outcomes much better than probability mass.

FWIW, my tests used only ranking. I used Kev to drive E2E test runs using agent-device, where Kev had to decide the next action - like pressing a button, or scrolling down etc, to reach a certain goal (UI state). It went surprisingly well, but 4B BF16 was significantly better than 8B Q8. The latter made some wrong decisions in every run while the former was almost at Luna precision.

Fascinating insight. I'm curious, how do you know this?

Plus, a lot of IP from a previous startup (that was acquired last year), Pi Labs, was used to help push the model to SOTA performance quickly. The startup was building scoring models long before Jev released

Surprised they aren't doing these sorts of one-off models with Microsoft Phi, which is intentionally smaller, but there's no reason Microsoft couldn't try to make a slightly larger Phi model with more capabilities...

IIRC Phi has a tiny context windows.

And also, it is nowhere as good as Qwen

> This is based on one of the smaller Qwen models, just like Cloudflare's Clef, Strands decider

As of this afternoon, we also have Gemma4-based variants of strands-decider at 2B, 4B, 12B, and 26B: https://huggingface.co/StrandsAgents

I don't know if Fabio (who's leading work on this model line) would agree, but if I was going to start over I'd probably pick Gemma4 as the base rather than Qwen3.5. But it's not surprising to see a lot of Qwen competition, and I totally agree it's nice to see the work open.

Yep, Fabio here, and I agree!

I been using gemma 26b moe model as the system one variant for my own uses and its great and far smarter then any alternatives at around 200ms, bigger is better for these things as long as you dont need lower latencies, only caviat is calibration, if you need calibration better use JEV.

> Great to see open weights (if not open source)

The insistence of naming it open weights as opposed to open source is getting ridiculous, and it's both irrelevant (i.e. no one cares in practice) and factually incorrect.

Weights are source in language models. Apache defines source as ""Source" form shall mean the preferred form for making modifications". That is precisely what's happening here. Everyone is using the preferred form for making modifications to these models (including the model creators themselves). A model is "created" at init time, and then "trained" by modifying the weights.

All these models are open source. What's not open sourced (with qwen et all) is the training code. So open source model, no training code. And that's ok. There are labs that release those as well. Apertus and Olmo series come with open source models, open source training and open datasets. Nemotron comes with open source models, open source training and some open datasets, while others are not published. And that's ok too.

The fact that you see all these models being modified (from AR completion models to "decision models") and re-released should be all the proof you need. That's what a license offers you. The right to inspect, run, modify and re-release a model. A license cannot (and never did) give you any other rights. OpEnWeIgHtS is silly.

I mean I don't have a strong opinion but if I used the phrase open source models there'd be 5 comments going in the other direction.

I'm happy to have and be able to serve these models and see the ecosystem thrive. And lots of open innovation is outside of weights anyway as DeepSeek repeatedly shows.

You used the right term. I don't know what the parent commenter is on about.

Open source has historically meant you could download the source and build your own binary. The appropriate analogue here, IMHO, to the build->binary process is training->weights.

> OpEnWeIgHtS is silly.

Dictionary.com defines source as:

> any thing or place from which something comes, arises, or is obtained; origin.

> Weights are source in language models.

Weights are source in the same way as any x86 binary is source.

You easily modify a x86 binary and change behaviour or examine the machine code instructions. You probably are not aware how easy it is to change the behaviour of a binary executable.

But, binary is not the preferred form.

Also if compiling literary costs millions of dollars, I would prefer the precompiled one.

Reply on news.ycombinator.com