Running Gemma 4 26B at 5 Tokens/sec on a 13-Year-Old Xeon with No GPU

Running Gemma 4 26B at 5 tokens/sec on a 13-year-old Xeon with no GPU

I managed to run Google's Gemma 4 26B model on a thirteen-year-old HP StoreVirtual server with no GPU. After an initial failure due to missing AVX2 instructions, I used Claude to diagnose the issue and patch the ik_llama.cpp code. The result is a functional local AI running at reading speed on ancient hardware, proving that modern models can work on legacy enterprise iron with the right engineering.

"Good with AI has quietly come to mean pays for a subscription. I think the real skill is different: knowing a model well enough to point it at a problem nobody packaged for you, and telling whether the answer it hands back is actually correct."