Five small models on one laptop beat majority voting by checking their sources
Claude was faster and cheaper. We took the scenic route anyway
Running Llama 3.2, Mistral 7B, Gemma 2, Phi-3 mini and Qwen 2.5 locally on an M3 Pro, the authors had each model answer questions and cite passages, then used Claude Sonnet 5.5 to verify whether those passages actually supported the answers. A fixed rule that only switches answers when a source backs the change lifted accuracy from 77.1% to 85.0% on 1,000 questions. The first version failed its own pre-registered harm test; a revised rule passed.
If a passage says Sydney is the capital of Australia, it supports the answer "Sydney," even though the capital is Canberra.
- tobblegobble
I've had really great results with cheap flash agents using a system that has a parent agent that spawns 2 sub agents. They remain active throughout the whole session and don't lose context.
The 2 sub agents get the exact same prompt and tools as each other and they independently answer but before responding back up to the parent they exchange their findings which must be backed up by a citation and then debate their findings and citations repeatedly until they can agree on correctness of all points. They are also heavily encouraged to never assume and always verify with the prompt and tools and guards where they can be realistically enforced.
It churns through tokens but tokens are crazy cheap with some of the plans out there. Could Opus one shot most of those same queries in like 1/4 time? Yep, but that's not the point for me.
DeepSeek Flash for parent then DeepSeek Flash and GLM 5.3 flash as the debating subagents gives best overall results in my experience.
- LoganDark
I don't understand how this submission title ("Claude was faster and cheaper. We took the scenic route anyway") has anything to do with the article. I read the entire article and I'm still confused.
"Verify, Then Vote", "When five models disagree", or both ("When five models disagree: Verify, Then Vote") would probably have made a better one.