Lightreason pushes Qwen3.6-35B to 6.4× faster on a single RTX 4090
6-11X speedup running Qwen3.6 on my RTX 4090, full repo.
A local inference lab called Lightreason runs Qwen models faster on one RTX 4090 by reusing prepared GPU work and checking multiple tokens together. The 3B model jumps from 63 to 139.6 tokens per second, while the 35B model goes from 7.93 to 51.08 tokens per second on a capped Python prompt. The repo includes a Studio for side-by-side timing comparisons and detailed benchmark evidence.
The larger model’s text weights occupy 64.56 GiB. The 35B runner keeps most expert weights in system RAM and moves the ones Qwen actually uses.
- shutty
Nice but it would be also great to include strata and llamacpp to the benchmark. Otherwise it’s 11x speedup over the baseline no one uses
- kennywinker
That's cool, but what are you doing that you want to run qwen3.6-35b-a3b, over qwen3.8-flash.next, which runs great on a 4090