PostHog skips the LLM call for 98.7% of session summaries

Not Everything Needs an LLM

PostHog skips the LLM call for 98.7% of session summaries

LLMs ramble and cost a fortune, so PostHog swapped in decision models like Jev for narrow, structured judgments. In Replay Vision, two yes/no questions about personal info gate each session summary, and 98.7% pass without ever touching Gemini. Decision models are up to 444.6x cheaper and 193.6x faster, but they judge only what's in front of them — multi-step reasoning still belongs to LLMs.

LLMs love to yap. Ask a simple question and you get a few hundred words of "it depends." It's annoying as hell.

More from this day

2026-11-17