Apple Silicon macOS VMs: 11–16× Faster LLM Inference with a Metal Capability Shim

Apple Silicon and macOS VMs: 11–16× Faster LLM Inference with Llama.cpp

Cua's Lume team reveals a process-scoped Metal capability shim that unlocks newer GPU paths inside macOS VMs on Apple Silicon, boosting llama.cpp inference dramatically. On an M1 Ultra, TinyLlama 1.1B prompt processing jumped 11× to 98% of bare-metal speed, and token generation 16×; Gemma 4 12B saw 7.2× and 14.5× gains. The shim adjusts two capability queries (Apple family and threadgroup memory) to let Metal apps select faster kernels, without true GPU passthrough. Source, benchmarks, and reproduction steps are included.