Janas streams 48 GB LLMs from SSD on a 32 GB laptop, beating llama.cpp by 8x
Janas: Run LLMs larger than RAM by streaming experts from SSD
Janas-LLM, a C engine by Maurizio Cammalleri, runs mixture-of-experts models larger than RAM by streaming experts from an NVMe SSD. On a 32 GB laptop, Qwen3-Next-80B (48 GB) hits 37 tok/s with drafts, 24 without, versus llama.cpp's 2.8; prompt reading reaches 62 tok/s against 16. It outperformed llama.cpp on all 13 models compared in Q4_K_M. Experimental, Linux and Windows.
37 tok/s from Qwen3-Next-80B (a 48 GB model) on a laptop with 32 GB of memory, 24 without drafts, where llama.cpp manages 2.8; reading a prompt, 62 tok/s against 16.