Homebench: Benchmark Your Local LLMs for Speed, Memory, and Quality in One Command
Homebench – Benchmark local LLMs for speed, memory, and quality
Homebench is a single-command TUI that benchmarks local LLMs from Ollama, LM Studio, llama.cpp, vLLM, or any OpenAI-compatible server. It measures tokens/sec, time-to-first-token, memory footprint, and runs a 31-task quality suite with deterministic grading. The tool features a live leaderboard, response caching for fast re-runs, custom task packs, history and diffing, batch throughput testing, and a hardware fit checker that suggests which popular models your machine can run. No config or API keys needed—just pip install and run.
There are great tools for one half of this problem, but nothing local-first that does both: llama-bench measures speed only, and lm-evaluation-harness measures quality but has no polished laptop UX and isn't built around the model runners most people actually use locally.