Unsloth Dynamic 3.0 GGUFs: >10% Better Accuracy at Same Size
Unsloth releases Dynamic v3.0 GGUFs for Qwen3.8-27B, claiming over 10% better top-1% accuracy at the same size compared to other providers. The new quants use a higher-quality imatrix calibration dataset, improved layer selection, and post-training quantization (no QAT/QAD). They also introduce smaller 1-bit quants and remove the MTP module from smaller sizes to save disk space. Benchmarks show significant improvements in KL Divergence and Divergence-300 @32, with no overfitting.
Our new methodology composes of many new features and improvements. We now use a much higher-quality imatrix calibration dataset from diverse sources.
- walrus01
It would be nice if unsloth published GGUFs would use a version number or something, because now I have multiple different files on local storage that otherwise have exactly the same name.
"Qwen3.8-27B-UD-Q8_K_XL.gguf" for instance.
The one downloaded at least 4 days ago is a different thing and is NOT the "Dynamic 3.0" GGUF which I am now downloading, which I presume will have a different sha256 checksum?
The unsloth page says dynamic 3.0 is released "today", but I have an older copy of qwen3.8 27B Q8 which I downloaded, if I remember right, at least 4-5 days ago...
- Alephinitesimal
I mostly use local models when the data has personal information. Earlier this year, I felt the coding quality was still not as good as Claude Code.
One thing that works for me is to ask the local model to make some fake data with the same format, let Claude Code work on the fake data, and then bring the code back and run it locally on the real data.
This way the real data never leaves my machine, but I can still use a stronger model for most of the coding.
- xlayn
Hey Unsloth, your gguf are the first ones I look for when I want to download a gguf model.
Today I was trying in fact to see, what's the smallest Qwen3.8-27B that I could run and get good results, say restricting it to 16GB of ram.. so I went, pick up the Qwen3.8-27B-UD-IQ2_XXS.gguf and them BAM, error on MTP... now I understand why after reading your announcement.
Beyond the space saving, why removing the MTP? improves speed exactly for the group that could benefit from it.
- spider-mario
Will this also be applied to older models like Qwen3.6? 35B-A3B still has its uses with its higher speed than the dense 3.8 27B.
- johndough
Are there benchmarks for the various Qwen3.8-27B quants that actually measure writing code, maybe even with multiple steps? Low KL divergence does not mean much when the model gets stuck in doom loops all the time.
I could of course download and test myself, but that would take days with my internet connection.
- jwr
These are very good!
I'm hoping for speed improvements because the only problem running the 27B model on my Macbook pro (M4 Max) is the speed: 20 tokens per second. I benchmarked and MTP actually makes things slower, so I disabled MTP altogether. I'm hoping there will be some breakthroughs or optimizations that will allow me to run this at 30-50 tokens per second, which would make a big difference.
- throwa356262
"We also made some smaller UD-1bit quants with UD-IQ1_S being 6.2GB (without MTP) which retain around 72% top-1% accuracy yet being 89% smaller"
This is crazy! But has anyone tried these lower quants on real projects?
- ankushdograuk
Waiting for MLX version