Hacker News Reader

Comment by UncleOxidant | Hacker News Reader

> Qwen-3.8-27B - the best thing that happened in AI so far.

Qwen3.8-Flash_Next should be in the running for that title as well. It's astonishingly good. I use both of them. If I need more memory on the machine for other things I run 27B, If I can get by with much less RAM I run QFN (takes about 102GB on my Strix Halo box with 262,144 context). QFN runs quite a bit faster on my hardware - about 50t/s. 27B with MTP starts out around 22t/s but degrades over time as the context fills up. Both are great models, but I'd give QFN a bit of an edge (at least for the coding projects I've worked on). But both feel on par with Claude Opus 4.6 - 4.8.

View on HN

Apparently Qwen 4 might land this week, and it uses the same architecture as 3.8-Flash-Next (so various LLM serving tools should be able to handle it already). I'm hoping Qwen 4 includes some smaller models - the last 4B and 9B from Qwen were for 3.5.

If they actually release the weights.

The head of Qwen just confirmed that 4 and 5 will remain open.

Open or "community license"

Doesn't matter for local use anyway.

I remember seeing somewhere credible that it was going to be Thursday this week (tomorrow?), but don't remember where.

Assume you've seen the Apsara conf announcement / the planned 'tiers'[0]. Do you have a planned use case for a 4b / 9b? Hopefully the overthinking will get better on the smaller models.

[0]: https://www.yottalabs.ai/post/qwen-4-max-vs-plus-vs-flash-vs...

I’m really wishing for a 35b-a3b version. It’s just such a great speed-capability trade off.

Agree, I run QFN on my AI Pro R9700 (32GB) using https://github.com/Niko1221/Strata It's even faster than on a Strix Halo. Typically I get around 90 tokens/sec with a small context, down to 70 tokens/sec when it's near the maximum context usage.

Err strata is just a fork of llama cpp with fancier uis, right?

Strata manages MoE expert offloading completely differently, which for QFN means ludicrous speed.

I run QFN I2 XS on my 3060ti + 48GB RAM for it. Get around 2x token with 200k Context window. Good enough for me.

QFN is obscene. It runs faster than 27B on all my setups, and it always outdoes it in my uses (from coding to translation to vision-related): from somewhat to a lot.

What do you mean “outdoes” — over complicates?

Google: Outdo is a verb meaning to do something better than someone else, or to surpass someone or something in quality, quantity, or performance.

Surpasses. At least in my case, 27B tends to waste more time in the reasoning stage, too, so 27B overcomplicates. QFN just does a better job. Sometimes I eyeball, sometimes it's measurable like translation chrF++ 59.3 vs 67.5.

Which quant of QFN are you running on your Strix Halo?

HF is telling me that IQ4-XS is the largest one likely to fit without much contortion.

That's the one I'm running using a fork of the gufo inference engine as gufo upstream doesn't support that quant.

I'm running a regular Q4, what about that quant format justifies the fork?

I switched back to glm-5.3-flash from qwen3.8-flash-next.

qwen was faster in t/s, but feels like glm spends less tokens thinking and seems better at finding issues, to the point where it ends up being faster.

Same experience, the 4bit quantized xlarge flash next runs at around 70t/s across 3 radeon v620s. Still have to retry the MTP version with full resolution.

> Qwen3.8-Flash_Next should be in the running for that title as well.

Unfortunately it still hallucinates, which limits it's usefulness for real world tasks. :(

Every model hallucinates, if not given the right context and tools.

How do you tailor your context and what tools do you use?

give it information it needs to complete it's task, and not more so it doesn't get confused.

I find they hallucinate even with the right context and tools.

So, while useful for some people, for the things I care about they're presently pretty useless. :(

Reply on news.ycombinator.com