Hacker News Reader

danielhanchenon Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Sorry on the delay - oh haha that would be cool :) We did release 2bit dynamic ones, but unsure if they'll be helpful

danielhanchenon Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Yes we do! Sorry on the delay

danielhanchenon Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

We use Duck Duck Go - sorry on the delayed response as well

danielhanchenon Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Thank you and appreciate it! Sorry on the delayed reply as well

danielhanchenon Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Oh yes LM Link is cool!

danielhanchenon Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Hey sorry on the delay - we just added API support, so you can access a remote server - it includes optional python, tool call, bash and web search support if you enable them.

For SSH - we haven't yet done that - for now we have a SHA256 encryption approach, but it's not SSH yet. HTTPS will also sadly have to be the end user's setup process as well - we plan to make it better soon!

danielhanchenon Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Hey! Sorry for not replying sooner - yes we'll keep publishing more KLD - sadly some are saying we are "optimizing" for KLD now since we posted so many haha - but the whole purpose of quantization is to match the BF16 logits as much as possible whilst reducing disk space (ie reduce KLD).

In general so this is funny and a quirk of quantization - sometimes 8bit, 4bit models do BETTER on downstream benchmarks (SWE Bench for eg), since sometimes rounding can actually somehow act as a "regularization" method (this is just my hunch).

So KLD isn't that expensive, since we leverage the trick of causal attention - since causal attention is lower triangular, we can do 1 forward pass on the enter text (say 2048 tokens), and you attain logits for the prediction for every token's position - so this is O(N^2).

However coding benchmarking require actual inference, and cannot use the causal attention trick, and it's best to run them 10 times since temperature = 1.0 is not deterministic - and take an average. We plan to maybe do something like https://marginlab.ai/trackers/claude-code/, which takes a random sample and does it over time.

danielhanchenon Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Hey so sorry didn't reply sooner - yes the docker used to be I think 4-8GB ish since CUDA sadly itself is 4GB I think, and PyTorch takes the rest. So unfortunately the Unsloth Docker image has ballooned due to this. We tried reducing it as much as possible, but it's hard :( https://hub.docker.com/r/vllm/vllm-openai/tags for eg is around 11GB ish, ad we're 13.6GB ish.

We'll try our best to compress it more, but it's tough

danielhanchenon Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Apologies as well didn't reply sooner - Studio supports AMD out of the box now! We worked with AMD to make it work! One thing that is still missing is pre-compiled AMD ROCM binaries, which we're trying to see if we can integrate that.

Interesting on diskpart - let me check and get back to you [EDIT] - visual studio build tools, python 3.13, git, cmake, node.js are all msi-based installers - so these are likely the culprits on using diskpart - essentially MSI installers check if there's enough disk space before installing items

danielhanchenon Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Oh my apologies I didn't respond - if only HN had a notifier haha

Oh yes we added a custom folder button which can pull .gguf files for now from any folder - it supports LM Studio and Ollama ones - but afreed it's still a mess.

One of the goals is to somehow quick search for .gguf folders, and add recommended folders - we currently have folders for Ollama and LM Studio for eg

danielhanchenon Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

We made Unsloth Studio which should help :)

1. Auto best official parameters set for all models

2. Auto determines the largest quant that can fit on your PC / Mac etc

3. Auto determines max context length

4. Auto heals tool calls, provides python & bash + web search :)

danielhanchenon Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Haha :)

danielhanchenon Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

Haha :) We had some issues with Kimi-2.6 since it was int4 and we were investigating how to handle it :)

danielhanchenon Qwen3.6-27B: Flagship-Level Coding in a 27B Dense Model

We also made some dynamic MLX ones if they help - it might be faster for Macs, but llama-server definitely is improving at a fast pace.

https://huggingface.co/unsloth/Qwen3.6-27B-UD-MLX-4bit

danielhanchenon Qwen3.6-35B-A3B: Agentic coding power, now open to all

Yes sadly CUDA 13.2 is broken - NVIDIA will push a fix in CUDA 13.3

danielhanchenon Qwen3.6-35B-A3B: Agentic coding power, now open to all

Yes we have started doing diffusion GGUFs but it's in it's infancy :) But yes we do generate images to test quants out!

danielhanchenon Qwen3.6-35B-A3B: Agentic coding power, now open to all

Love the JPEG analogy :)

danielhanchenon Qwen3.6-35B-A3B: Agentic coding power, now open to all

Yes so chat templates and the actual implementations

danielhanchenon Qwen3.6-35B-A3B: Agentic coding power, now open to all

Thank you! Agreed on chat template issue

danielhanchenon Qwen3.6-35B-A3B: Agentic coding power, now open to all

Oh yes! This only applies if one uses hf download / snapshot_download - other normal download methods sadly won't have XET

danielhanchenon Qwen3.6-35B-A3B: Agentic coding power, now open to all

Probably yes

danielhanchenon Qwen3.6-35B-A3B: Agentic coding power, now open to all

Yep agreed at least 1 week is a good idea :)

We do get early access to nearly all models, and we do find the most pressing issues sometimes. But sadly some issues are really hard to find and diagnose :(

danielhanchenon Qwen3.6-35B-A3B: Agentic coding power, now open to all

Oh that is pretty good! And the SVG one!

danielhanchenon Qwen3.6-35B-A3B: Agentic coding power, now open to all

They sometimes do! Qwen, Google etc do them!

danielhanchenon Qwen3.6-35B-A3B: Agentic coding power, now open to all

Sadly it's not always chat template fixes :( But yes we now split the first shard as pure metadata (10MB) for huge models - these include the chat template etc - so you only need to download that.

For serious fixes, sadly we have to re-compute imatrix since the activation patterns have changed - this sadly makes the entire quant change a lot, hence you have to re-download :(

danielhanchenon Qwen3.6-35B-A3B: Agentic coding power, now open to all

Oh for multi files? Hmm ok let me check that out

danielhanchenon Qwen3.6-35B-A3B: Agentic coding power, now open to all

Hey thanks - yes agreed - for now we do:

1. Split metadata into shard 0 for huge models so 10B is for chat template fixes - however sometimes fixes cause a recalculation of the imatrix, which means all quants have to be re-made

2. Add HF discussion posts on each model talking about what changed, and on our Reddit and Twitter

3. Hugging Face XET now has de-duplication downloading of shards, so generally redownloading 100GB models again should be much faster - it chunks 100GB into small chunks and hashes them, and only downloads the shards which have changed

danielhanchenon Qwen3.6-35B-A3B: Agentic coding power, now open to all

Oh hey - we're actually the 4th largest distributor of OSS AI models in GB downloads - see https://huggingface.co/unsloth

https://unsloth.ai/docs/basics/unsloth-dynamic-2.0-ggufs is what might be helpful. You might have heard 1bit dynamic DeepSeek quants (we did that) - not all layers can be 1bit - important ones are in 8bit or 16bit, and we show it still works well.

danielhanchenon Qwen3.6-35B-A3B: Agentic coding power, now open to all

Yep we can do that probs add a table - in general be post in discussions of model pages - for eg https://huggingface.co/unsloth/MiniMax-M2.7-GGUF/discussions...

HF also provides SHA256 for eg https://huggingface.co/unsloth/MiniMax-M2.7-GGUF/blob/main/U... is 92986e39a0c0b5f12c2c9b6a811dad59e3317caaf1b7ad5c7f0d7d12abc4a6e8

But agreed it's probs better to place them in a table

danielhanchenon Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All

Thanks!