Why You Only Need the Frontier Model for One Single Edit
I discovered that splitting tasks between expensive and cheap models often increases costs because reading code is the real expense. Instead of handing off a plan, I found that swapping to a cheaper model immediately after the first edit saves money while maintaining performance. This approach avoids duplicating the costly context reading phase that traditional planning strategies miss.
The expensive part of an agent's day is not the fixing, building, or even the thinking. Opus fixing things does not cost money. Opus reading things costs money.
- robbie-c
I'm begging people to write articles themselves rather than letting Claude do it for them.
I want an expert opinion, if I just wanted to ask an LLM I have my own.
How can this article not mention the KV cache even once?
- nxtfari
This is really smart, like the author said, old idea but cleverly applied.
In case anyone wants a summary: don’t one shot, don’t use plan mode and hand off the plan to cheap executors, ask the frontier model to explore, create a todo list, and then start when it feels confident; stop it after first code edit, then prefill the context to cheap executor to continue.
- Too
> The mistake is upstream of the architecture diagram. People price agents the way they price people: senior time is expensive, so minimize senior involvement.
> But the expensive part of an agent's day is not the fixing, building, or even the thinking. Opus fixing things does not cost money. Opus reading things costs money.
Heh, another rediscovery that agents and humans are alike. By the time I've researched a Jira ticket and made it unambiguous enough to outsource, I might as well have written the code myself instead.
- unholiness
No mention of KV cache, one of the biggest reasons not to switch models mid-stream. Once you pay, say, 50k input and 50k output Opus 4.8 tokens, you don't pay token cost for cache reads of the 100k context while it builds. Switch to another model, you'll start with the cost of 100k input tokens on the smaller model to get that context loaded and its unique KV cache set.
The post may have some real insight here, where this 100k working context is actually better than trying to summarize it's findings into a plan. It's also right that the handoff is a perfect time to edit the context (removing planning instructions). But it doesn't mention it's a trade-off: the plan is smaller, so it's a cheaper "on-boarding" of the next model. Seems quite plausible that this is worth it for 1-off tasks. If it's right, this is basically "plans are useless, planning is everything" for LLMs.
My problem is, I think the plans are useful. I want to review and edit them. I want them to give context for the upcoming code review (even if humans aren't reviewing). LLMs are notoriously bad at explaining why they're doing something in the moment. Humans are notoriously bad at accepting there's no reason why. I think the humans have it right here, and to bridge this gap want my PRs, my docs, and my comments teeming with reasons why. Plans help with that.
- figmert
The reason I use plan then implement is because I can adjust the plan, whereas if I get it to implement straight away, it might (and often does) make the wrong decisions that will be harder to adjust, or I'd have to adjust it after the fact.