Shoehorn - Quantize any model to fit your machine's memory
Show HN: Shoehorn – Quantize any model down to run on your machine
Shoehorn is a tool that makes any language model fit your machine's actual memory. Instead of preset quantizations that waste headroom or fail at load time, Shoehorn measures your available memory, subtracts inference overhead, and solves a per-tensor mixed-precision assignment to use up to 99.99% of your budget. It integrates with llama.cpp and offers a local web app where you can discover models ranked by quality, fit them with one button, and chat. Available via Homebrew or Cargo.
Shoehorn starts from the memory you actually have, subtracts what inference itself needs, and solves a per-tensor mixed-precision assignment that lands within a rounding error of the remainder — routinely using 99.99% of the budget, sometimes to the byte.