Apple Silicon macOS VMs: 11–16× Faster LLM Inference with a Metal Capability Shim
Apple Silicon and macOS VMs: 11–16× Faster LLM Inference with Llama.cpp
Cua's Lume team reveals a process-scoped Metal capability shim that unlocks newer GPU paths inside macOS VMs on Apple Silicon, boosting llama.cpp inference dramatically. On an M1 Ultra, TinyLlama 1.1B prompt processing jumped 11× to 98% of bare-metal speed, and token generation 16×; Gemma 4 12B saw 7.2× and 14.5× gains. The shim adjusts two capability queries (Apple family and threadgroup memory) to let Metal apps select faster kernels, without true GPU passthrough. Source, benchmarks, and reproduction steps are included.
The guest's conservative answers hid a surprisingly capable GPU path.
- simonw
It looks to me like this won't speed up llama.cpp for everyone, just for users running it in this particular kind of Virtualization.framework VM.
The fix here works around a problem where the VM was causing llama.cpp to select the wrong kernels.
- engzaanin
That makes sense. The title initially sounded like a general llama.cpp speedup on Apple Silicon, but if the improvement comes from fixing kernel selection inside Virtualization.framework VMs, that distinction is pretty important.
- aeriose
What I don't get, which this article doesn't talk about, why would Apple’s Virtualization.framework expose a lesser Metal profile instead of reporting all capabilities supported by the host GPU?
- thehamkercat
> 11.08× faster and generated tokens 16.36× faster than the same workload in the same stock VM.
So this was the comparison, for me the title was a bit confusing
- w10-1
Related: does anyone have a basis for guessing whether the Neural Accelerators found in M5 Pro+ (accessed by Metal 4) will make their way into the M6 base processors?