I Burned All My Tokens Researching How to Save Tokens

TFA says "no hallucinations" but you can't fix hallucinations with rules or other models. I know I'm screaming into the void but whatever.

Simply starting in the right part of the search space is the biggest predictor of success.

The best way to save tokens is to start out the deep research pass with cheap models and then funnel the findings through increasingly powerful models. I've got a pipeline right now that uses all 3 of the gpt-5.6 model variants to address each stage of the process. If you are using models like sol or terra to generate hypotheticals and perform initial exploration, you are leaving money on the table.

5 hypotheticals out of luna will massively outperform 1 hypothetical out of sol, but the cost is the same and so is the runtime if you do it in parallel. The hypothesis generation phase is also a great place to mix and match models from different vendors. The more diversity at this step the better.

The other thing I started looking into is batch pricing which represents 50% off for OAI tokens right now. With some tweaks to the UI/UX of an enterprise chatbot, I think it is possible we could have users get comfortable with the idea that questions to the robot might take a day to come back. The key is that this has to actually work. Users don't mind trading time if their questions come back with high quality results.

I am not a proper developer and only use AI for faster research of topics so please forgive my ignorance. Could one not save a lot of money on tokens by using the 80/20 or 90/10 rule in that 90% of AI usage is on local models and save that last 10% or less for the frontier models where the local model did not meet the needs? Did they cover this and I misunderstood?

I built a deep research pipeline to investigate why my deep research pipeline costs so much. the answer was the deep research pipeline.

My own set of heuristics

1. Use less subagents. The act of using subagent already needs dumping adequate amount of context for subagent to work effectively

2. Refactor when file grows large. Reasonably scoped files is easier to understand, for both human and agents, and consume less tokens because less guessing which section of the file to read

3. More capable models for planning, learning. Cheap models for execution. Same for human teams, senior member for higher level planning while junior member more on the act of translation of idea into code

4. I wasn't doing any fancy stuff that opus can't handle

5. Consult a second opinion for things you are not sure. (I have made a skill specifically for that)

6. Audit how your habit correlates with token consumption. I found out a skill a use frequently is using a lot of context just so a subagent can get all the context. (made a tool to audit claude specifically)

You should consider testing our RemembrallMCP (note I'm the author). It substantially cuts token usage and you get better results in codebases.

https://github.com/roboticforce/remembrallmcp

Many of us were saying it a year ago but now with model restrictions (e.g. fable) and pricing changes it should be obvious to people that part of the economics is avoiding vendor lock-in with evals (so you can easily switch providers/models/optimize) and increasing control by investing in local models which could be good enough for your tasks, at whatever the price for your cloud compute is. Eventually consumer hardware will also be able to run good enough.

You can use Big/Cloud LLMs to help you "find good enough configs" for your local/small llms [1] and stay quite nimble in the face of rapid change.

- [1] https://alexhans.github.io/posts/find-the-loop-story-first.h...