GPT-6 Astra edits model weights directly — but a human researcher could do the same
LLMs are not superhuman at reading and writing neural network weights
A recent paper claims GPT-6 Astra can read and rewrite neural network weights, including fixing a 0.8B Qwen model that looped after 32k tokens on math problems. Sean Goedecke argues this isn't superhuman: the toy game networks had labeled input channels, making the task trivial with a few lines of Python, and Astra's Qwen fix followed the standard abliteration playbook — sample contrasting texts, measure contributing weights, trim them. Direct weight editing is a neat trick, but training is still the right way to improve models.
It's cool that Astra was able to make direct weight updates. But we shouldn't think of this like a radically new capability that will change how AI research is done.
- nfw2
One could say they might be "superhuman" here in the same way that LLMs might be said to be superhuman at math. Perhaps they do this work faster and more effectively than any person could manually. But they are certainly still doing it the "human" way ie reasoning with legible and coherent thoughts, rather than communing through some Matrix-like symbology which is what people who skimmed the article seem to be assuming.
- nagi_builds
"I think a smart human ML researcher could perform all of these tasks."
And do that for how long?
- ChrisArchitect
Discussion on source: