Picchio: Exposing Silent CPU Fallback and Misleading Speed Claims in Local LLMs

Same model, same Q4_K_M label: 5.02, 5.07 and 5.27 bits per weight

I built Picchio, a simple Python tool that catches silent CPU fallbacks and exposes misleading speed claims in local LLMs like llama.cpp and Ollama. By measuring actual GPU usage against engine logs, it reveals why your model might be running slower than expected, even when labels suggest otherwise. This one-file utility helps you verify if your GPU is truly doing the work.

"Most GPU speed claims are one tok/s number. That number can be correct and still tell you the wrong story."