Jevstiller - Local model with guaranteed agreement to Jev
Show HN: Jevstiller – Distill Jev into a local model, with a disagreement bound
Jevstiller learns a small local model from Jev's own answers, answering what it's sure about in ~15ms on CPU. Set a target agreement (e.g., 98%) and Jevstiller guarantees that at least that share of requests get the same label Jev would have returned. It uses a statistical bound to ensure the contract holds with 95% probability, even under continual retraining, with a permanent audit and drift detection. Ideal for agent loops, game ticks, or any low-latency classification.
Set one number, say 98%. Jevstiller returns the label Jev would have returned on at least that share of requests.
- gingersnap
Is the local model similar to model2vec?
- tgluck
Author here. This puts a proxy in front of repeated Jev classification calls. At first everything goes to Jev; from Jev's answers it trains a small head on frozen sentence embeddings, picks a confidence threshold with an exact finite-sample bound so that at most 2% of all requests get an answer Jev wouldn't have given, and then answers the confident share locally at ~15 ms on a CPU. A permanent 2% audit keeps checking; if agreement breaks, everything falls back to Jev and it retrains.
Known limits: agreement is not accuracy (if Jev is wrong, so is the local model); coverage tracks how consistent Jev itself is (22% on noisy tweet tasks, 80% on news); it speaks Jev's API only, an OpenAI-compatible front is on the roadmap. Since 0.4.0 the guarantee can also cover "would Jev have been unsure", which matters if your code routes low-confidence answers to review. Apache 2.0.