Models Are Getting Dumber on Purpose
Reasoning scores are climbing while per-token compute shrinks, but the trade-off is deliberate: labs are sacrificing world knowledge for reasoning skill. Facts take space in weights, rot quickly, and are hard to fix, while procedures compress well and stay fresh. The future points to small, reasoning-focused models running locally, with knowledge supplied by the harness at runtime—a shift that could largely solve hallucination.
A wrong fact in a knowledge base is an ordinary data bug, the kind we already know how to trace, fix, and write a regression test for.
- kennywinker
Ideally what I'd like to see is pluggable knowledge bases.
So if I'm e.g. coding a SwiftUI app for navigation, I'd take 9B of basic coding and reasoning, add 10B of swift/swiftUI, add 5B of GIS/geography knowledge and another 5B of frontend app design knowledge. My model doesn't need to know a single line of python.
Then when I want to research electronics components, I grab a 15B model of agentic research techniques, and add in 10B of electronics knowledge, etc.
I don't want general purpose models. They try to be everything to everyone. I want to click together a model that is laser-focused on what I am doing, and I want to run it locally
- COAGULOPATH
This AI generated post (100% on Pangram) is pretty out of date.
>On SimpleQA, a benchmark of factual recall with no tools allowed, the current leader is Gemini 2.5 Pro at 53%, so the best recall money can buy still misses half the questions.
SimpleQA hasn't been updated in a long time. Gemini 2.5 Pro is a sixteen-month-old model, not "the best recall money can buy".
>The part I find most promising is what this does to hallucination. When a fact lives in weights, a wrong fact is unfindable and unfixable.
This seems confused. LLM hallucinations don't come from the weights containing "wrong facts", they are artifacts that appear at runtime.
>When the fact lives outside the model, a wrong answer has an address. The model cites a document, so you can open the document. If the document is wrong, you edit the document
You can make any modern LLM explain its reasoning and find sources for its claims. None of this has anything to do with facts needing to exist in weights or in harnesses.
The internet is full of wrong information and I cannot magically edit it to make it all correct, so this doesn't help me.
>if a model is factually wrong a claim with a source is checkable and a claim from weights isn't.
Why? If a model's weights claim that Bart Simpson became President in 2020, why does this fact suddenly become uncheckable?
- msdz
Great article, even if it will be interesting to see whether things continue to develop in such a direction or not.
> There's a version of this future where the model card stops listing a knowledge cutoff at all, because what's left in the weights goes stale on a scale of years instead of weeks.
Future?
Even just recently I’ve read of two approaches to this problem:
Cactus have come up with Needle [0][1], which is their tool-calling focused 14 MB model (still an LLM!) – no world knowledge engrained.
And instead of say, tool call structure, VibeThinker [2][3] focuses on reasoning over world knowledge.
Combine these two approaches with a reliable search tool/a safe way of accessing the internet for the model, and you’ve got a probably slightly slower model for factual questions, which on the upside however doesn’t hallucinate.
[0] https://cactuscompute.com/needle
[1] https://news.ycombinator.com/item?id=49246804
- pulkitsh1234
But is Reasoning and Facts truly separate ?
To reason properly about the human condition (eg. World War) wouldn't you need to reason on some facts ? And then reason how some "facts" change the human behaviour ? How can you arrive via pure reasoning to predict how a collective of humans act ? We are not reasonable, humans are not logical deterministic machines confined to algebraic rules.
- hypfer
Man, both that Blogpost and some comments here read like science-fiction.
Specifically, creative writing driven by nerds dreaming about a future, without proper grounding in reality, constraints and all that stuff.
Which is kinda ironic given the topic.
And also important to do, because we should keep dreaming. We should just also be aware of when we are doing that and mark it as such.
- kaufmann
I think the idea is reasonable, however the SimpleQA Bench stopped measuring in Sept. 2025.
So newer data would be interesting.
(It seems a bit like an AI generated argument that uses old facts - something that happens to me quite often)
- Animats
This makes hallucination detection more important.
There's no reason that an LLM should have a vast number of obscure facts encoded. It can go out to a search engine for such facts. But the LLM has to be clear on what it doesn't know.
(Google's pricing for search from programs starts at $2.50 per 1,000 queries. If an LLM reaches out to Google, it has to pay.)
- zmmmmm
It's a nice idea conceptually to decouple knowledge from reasoning but like everything in life I think it's something of a fantasy. When you ask a model to do something, its response is grounded in all the world knowledge it has from those facts. If I ask it to color a circle blue and then red, it knows what blue and red are, that they are colors, that they are different, and this is essential to complete the reasoning about that task. The idea of abliterating all the "knowledge" but not affecting the reasoning pretends there is a clean boundary between them that isn't there. Yes if you take it to the extreme - does it "know" Einstein's birthday - it is very stark. But I think the boundary to where general knowledge starts to interplay with broad reasoning is not nearly as far from those specific facts as people assume.