885 karma · joined August 5, 2024
Then the UK voted for Brexit. Scotland and Northern Ireland voted against but England voted for it, and it outweighs both, so the UK left, dragging Scotland and Northern Ireland along and out of the EU. Neither were hugely amused.
So to answer your question, the Venn would likely be somewhat disjoint. That's a simple summary of a messy bag of thorns, though, so the details are likely way more nuanced.
I use the Unsloth UD_Q2_K_XL GGUF with default parameters, along with that custom template linked elsewhere in the thread, and no K/V cache quantization.
Where Opus has seen it before and knows how to do it, Qwen knows how to work it out. It turns out it's surprisingly capable at working things out. The obvious drawback is that it takes tokens and time.
All the same, getting to run something this capable locally is momentous, and suggests to me that streaming tokens from colossal data centers might not be the long term path forward.
Interestingly, it also seems to tend toward self-correcting, which makes lower quantizations borderline usable. There'll be more faffing around, but still converging toward a solution. I wonder if that's a deliberate product of its RL.
It's both funny and a bit terrifying and I still can't quite believe it. It runs decently on a gaming PC! Opus 4.6 came out only 6 months ago and was then broadly considered the new SOTA by a comfortable margin! How in hell did they package capability in the ballpark of a Feb 2026 frontier SOTA into 27B?!
More importantly, what's the point of building monster-scale data centers on unprecedented amounts of debt when a more than good enough model runs on a GPU from a couple years ago?
The coming months are going to be exciting, that's for sure...
"The models at this size continue to get better at an impressive rate. We don’t need to spend half a million dollars on datacenter-class hardware just to run a competent model."
Worth noting as well that the weights come with an MTP layer that seems particularly accurate, to the point it can give you up to 6-8 correctly predicted tokens consistently enough to be useful. Which obviously boosts its speed enormously.
I find it difficult to believe how good this model is. It feels like it's lagging heavyweight frontier models by less than a year, and it runs on your PC.
(Which I still think is bonkers, FWIW.)
Opus 4.6 is an especially interesting comparison point, I think, because it was a step change; IMO it's when LLMs became serviceable for coding. Yeah, pre-4.6 models did output code, and that code often superficially worked; and bringing it up to production standards still generally meant rewriting it entirely. Opus 4.6 is when that changed.
From my early tests, it's looking like the public benchmarks are not misleading, and Qwen 3.8 somehow got there too, by and large. I've got a few personal tests. One is a mid-complexity one-shot, purposefully underspecified. Beyond a few minor bugs that it could easily fix once pointed out, Qwen 3.8 largely aced it. There are a bunch of things I'd improve, but that was true of Opus 4.6's output too, and by and large the code is clean and well structured. Also worth noting that I'm running Qwen 3.8 fairly aggressively quantized to fit in VRAM; I'd expect tighter results still from the full weights.
Another test I ran is a variant of a common puzzle with an additional structural constraint that makes the usual solution inapplicable, so the model has to actively turn away from the well-known solution and construct a new one that takes the constraint into account. I've never seen a home model pass that test. Kimi K3 passes it, GLM 5.2 passes it (painstakingly). Qwen 3.8 struggles a lot... but does arrive at a correct solution. First time I see a home model do so.
I haven't yet tested it on long multi-turn scenarios. In my experience, that's where pocket models are weakest against heavyweight ones, especially when quantized.
That aside, it does seem like Qwen 3.8 can, in fact, trade blows with Opus 4.6. I don't know yet if it could replace it, and my money would be on no, but I may well be wrong about that considering how weirdly capable it is.
Interestingly, Qwen 3.8's MTP layer is uncannily accurate too. It still gave me good results up to 6 to 8 predicted tokens, which boosts its speed so much it's competitive with Qwen 3.6 MoE. So that's another bizarrely impressive thing about it.
And given all of the above, I do think that the trillions of dollars invested into OpenAI and Anthropic are becoming harder and harder to justify.