Reame - CPU-first LLM inference server that accelerates with use

Show HN: Reame – a CPU inference server that gets faster as it runs

Reame is a lean, fully-tested LLM inference server built on llama.cpp, specifically designed to maximize performance on affordable CPU hardware like shared vCPUs and free tier ARM boxes. Unlike traditional servers, Reame treats low-cost CPUs as first-class citizens by utilizing a persistent disk-based KV cache and self-regulating speculation to ensure the same computation is never performed twice. It excels in narrow, repetitive workloads such as document extraction and batch processing, offering an OpenAI-compatible API that allows developers to run private, sovereign AI models at zero marginal cost after the initial request.

"On a CPU, never compute the same thing twice."