For someone who's been running local models for a long while, these are very very exciting times.
For someone who's been running local models for a long while, these are very very exciting times.
I'm surprised the 26B-A4B is better? It should be faster too, interesting. I'm excited to try 31B with MTP, because MTP-2 is what makes 27B bearable on the GB10.
What are you using it for? Agent-based coding, or something else?
For coding I don't need image support so I stuff the entire GPU with text-only mode. I don't have a workflow where I send LLMs off to generate thousands of lines of code but what little coding I did I did with Qwen3.6 and it was spectacular, as you likely suggest.
However I find qwen unbeatable for toolcallling. I think gemma wasnt trained on that at all.
The AI space moves so fast! I'll check it out again.
There are definitely differences in the eagerness to tool-call that you'll need to manage. And for all local models I've ever used, I've had to micromanage the tools provided by servers to eliminate any possibility that they reach for something wonky or confusing.
Gemma4 chat template seems to had multiple issues, at least with llama.cpp, not sure they're all fixed yet. It assumed simple types for parameters for example.