Frugon - Local open-source LLM cost analyzer
Show HN: Frugon – Find which LLM calls a cheaper model could handle (local, MIT)
Frugon is a free, local, and open-source tool designed to analyze your LLM call logs and identify exactly where your bill is leaking. By running entirely on your machine, it ensures your data never leaves your control while calculating potential savings from switching to cheaper models. Users can capture logs via a local proxy or upload existing JSONL files to receive specific routing recommendations. Frugon helps developers and AI teams optimize costs by suggesting which calls can be handled by more affordable models without sacrificing quality, all while maintaining strict privacy with no external data transmission.
Your data never leaves your machine. Your keys go straight to your own providers. Nothing reaches us.
- indigodaddy
What we need is an AI gateway/router that will actually first analyze the input tokens and then decide what model to use. If it's so simple that a dirt cheap qwen 3.5 flash or whatever will be fine, then it chooses that. If it deems we need GPT 5.6, then it uses that, etc. does anything like this already exist?
- westurner
Evals and OpenInference (OpenTelemetry) might be useful.
Costed opcodes (like the shelved eWASM opcodes cost chart) would be useful for this model routing problem as well.
Is this the cost to converge problem, the minimize cost to converge upon sufficiently low error problem, or the minimize cost and error problem?
EA methods: mutation, crossover, selection
Gradient descent as a mutation, crossover, and selection pattern; back up when the error/cost stops decreasing for too long and try a different branch.
A simple experiment: vary only a nonce in the prompt and compare output value. The nonce is a parameter. The model is a hyperparameter.
- isadubois
This looks super clean. I'm curious about the --judge command. How does it evaluate if the cheaper model's response is a "tie" or acceptable? Is it using a specific LLM-as-a-judge prompt template?