How Castform + Neon Beat GPT-5.6 Sol on Retrieval at 100x Lower Cost
Beating GPT-5.6 Sol on retrieval with 100x cheaper open models
A new post-training platform called Castform, combined with Neon's Lakebase Postgres and Search, lets developers fine-tune open-source models to outperform frontier models like GPT-5.6 Sol on agentic retrieval tasks, while cutting costs by 100x. The key is using RL post-training to teach small models to search and cite sources effectively, with Neon's dynamic scaling handling bursty workloads and branching enabling isolated training environments.
Most teams' best training data is just sitting in their databases. The problem is that turning raw data into something usable is hard, and letting agents read, search, and mutate data cheaply at scale requires advanced infra. Pointing Castform at Neon skips both.
- softwaredoug
People are building agentic search one of three ways:
1. Actually good retireval. There’s been a lot of progress on serving the kinds of queries agents tend to serve, from places like Hornet, MoxedBread, LightOn. Particularly in late interaction
2. Smarter harnesses with models/judges validating the result. This is now just seen as the generator/ evaluator pattern. Here’s where people try to just use grep or some other naive retrieval system. Let the agent figure it out. But it’ll consume a lot of tokens to get good results as it iterates and loops.
3. A model trained for retrieval. Give it dumb retriever like in (2) but it is fine tuned on the task as in (1).
This article is 3. But we’ve been seeing this all year with SID.ai, Gleans Waldo model etc. if this interests you I’d check those out, particularly SID.
I wrote about these 3 approaches here
https://softwaredoug.com/blog/2026/06/08/three-kinds-of-agen...
- mrinterweb
There is so much opportunity for purpose built models like this. Ideally a harness should spin up a subagent to offload to targeted models for specific tasks like this. I know this is not a novel idea. Claude code does some of this by handing off the "explore" agent work to haiku. I just love seeing that specialized LLMs are being developed.
- cmiles8
The big lab models are academically interesting but business wise they seem toast long term. There’s no way for these model companies to compete when the models are becoming a pure commodity and others offering options that are orders of magnitude cheaper.
It’s not that the big labs couldn’t theoretically just also put out 100x cheaper options but their business model requires them to generate huge revenues from higher priced tokens or they’ll implode.
- jillesvangurp
Models without tools and harnesses are not really that useful. My observation is that the tool ux is driving most progress at this point. There are of course open source tools and harnesses but they require more effort to setup properly.
The key challenge is to pick the right model for the right task or sub task and doing that automatically rather than manually. A big part of the problem here is that everybody is picking the most expensive and resource intensive models by default just in case they hit something that is a bit more difficult to get right. It's overkill. Most work people actually do is completely routine and would not have been a challenge for most of the mainstream OSS models.
I'm starting to suffer a bit from model fatigue. There are announcements almost on a daily basis about this or that new model. I can't keep up with that and I don't have time to try them out or evaluate them. I don't want to waste brain cycles on which one to use. I just want to get shit done without micromanaging AI models.
All this marketing BS and confusing naming isn't helping either. It seems a lot of that is just about tricking people into picking the expensive model so they'll burn through more tokens.
- aliljet
There is a more serious question in here that's not being answered. How effective is the retrieval in finding buried needles in larger and larger haystacks. And there's a correlary question, how effective could you be in finding paired needles in that haystack where you need to hold a needle to unlock finding another needle.
- BedVibe_Studios
This feels like the database equivalent of "use the right data structure." We've spent two years assuming the biggest general-purpose model should do everything. It makes more sense for retrieval, reranking, reasoning, and generation to each have their own optimized model if the routing cost is negligible.
- zkmon
Nice things like this, get "blue-washed" by the parent org's tech. For general adoption, kakebase should have been swapped out for a more generic non-databricks tech. 'Blue-wash' is a reference to an IBM's practice.
- linux_devil
Why do we need to train the model to solve for retrieval within the org, so we have to keep training it whenever new dataset is introduced , or am I missing something here ?