Running the 2.78 Trillion Parameter Kimi K3 Model on a Consumer Laptop
Run Kimi K3 using 29 GB of RAM at 0.50 tok/s
I built WASTE, a dependency-free C inference engine that streams activated weights directly from NVMe storage. This allows the full 2.78 trillion parameter Kimi K3 model to run on a standard 64 GB MacBook Pro without fitting entirely in RAM. By keeping only the necessary expert weights in memory and streaming the rest, we achieve local inference at roughly half a token per second, proving that frontier-scale models can operate on consumer hardware.
The interesting part is not the speed, it is that the whole thing is in the reachable range on a single consumer machine — and that from here the question is engineering rather than feasibility.