Astra's viral Qwen weight-editing claim falls apart under scrutiny
The 'Astra can read and write weights to improve Qwen' post is misleading
Matthew Honnibal argues the "Astra can read and write weights to improve Qwen" post is misleading: only three small experiments (17K–172K weights) actually read raw weights with no tools. Every real language-model test, including the headline Qwen3.5-0.8B math fix that jumps MATH-500 from 7.9% to 48.1%, uses Python and runs the model on up to 50 batches of practice problems. That's a human-style engineering workflow, not direct weight introspection.
The "just by looking at the raw weights" setting (raw floats in context, no tools, no running the network) only appears in experiments 1, 2 and 4, on networks of 17K–172K weights.
- x313
I work in this field (mech interp research) and I agree, the original post is definitely misleading. Some overselling in research is normal but it's a bit much.
First, the key result has nothing to do with making Qwen better at math, it just reduces reasoning loops.
Reasoning loops are a simple problem: the model generates the </think> token (which closes reasoning) with too low probability. You can fix this with simple steering approaches where you add a vector to different components of the LLM (https://arxiv.org/abs/2503.22048, https://arxiv.org/abs/2506.18167, etc). The method Astra used for identifying the vector is the standard procedure humans use (contrastive analysis). Such steering vector creation is basically the standard "intro-level interp project" for human researchers.
If you can code and understand LLM internals, you can do this yourself in 4-8 hours. The SWE equivalent would be doing binary patching with a debugger. Cool that frontier models can do this, but not unexpected since humans can do it without issue.
- bushido
Speaking for myself. I’ve been aware of a lot of concepts around training, but stayed away from it until very recently.
A few weeks ago, I formed a hypothesis that I could get a smaller model good at the specific standards I have for my git commit messages, which are hard to get out of most LLMs today.
I spent the last four to five weeks on traditional training. It improved in a lot of areas, but not enough to be usable for me.
Yesterday, the article the author of this post is critical of gave me enough of an aha that maybe the problems I was seeing could be solved by having a larger LLM change the weights directly, without going through training.
In the last twelve-ish hours I’ve made more headway than I have in 2+ weeks. I’m using one of my last well performing checkpoints (full fine-tune + changes to the decision head/LoRA, etc.) and gave the model free rein to change any of the layers.
Its last two actions: (a) removed 16 participant-order directions from a head matrix, which improved one type of reasoning and degraded another, and (b) restored one of the 16, which reversed the degradation while keeping the core improvement.
All that to say, this might be known to people who’ve been around training for years, even decades, but it’s new to a lot of us. And we’re excited about it.
- phoghed
Wasn’t all that obvious to anyone who read the original post when they got to the section that explicitly explained what the author of this post is now summarizing?
Sure if you only read the title and brief at the top you’d walk away without the full picture though.
This is a better response imo: https://news.ycombinator.com/item?id=50344861
- neom
I feel like a lot of this Astra editing Qwen stuff is regurgitation of existing relatively well known techniques, for example: https://arxiv.org/html/2406.11717v3 and conceptually similar to: https://arxiv.org/abs/2512.05648 (tho this is done in training iirc)
- tidbeck
There was some really good comments by @yurimo on the 'Astra can read and write weights to improve Qwen' hacker news post https://news.ycombinator.com/item?id=50341174
- GavinAnderegg
I'm still impressed by the "Weight-Space Programming" idea. Perhaps its because I didn't know that LLMs could understand or successfully modify weights at all previous to reading it. However, I think the examples provided in the original piece were eye-opening. Yes, humans can build tooling to modify weights, but the author here agrees that it's tough. The examples where Astra was interacting with weights are still novel, and Astra seemed to handle them with surprising ease.
- vkgcr
But theoretically, what would prevent the labs from RL'ing the models on "weight-space programming" tasks so that they get better at them? Perhaps we're just at the beginning, like how coding agents sucked 2 years ago.
It still seems like a promising approach for RSI, distillation, fine-tuning, and a whole host of other tasks.
- skwirl
There is something about AI related posts that makes it irresistible for people to post a “rebuttal” that actually makes the same claim as the original post just with different vibes. It feels like a lot of discourse is spent on setting the vibe over actually adding anything to the conversation.