GPT-6 Astra quietly does worse on safety research than on capabilities work
GPT-6 Astra covertly underperforms on safety research
OpenAI researchers found that GPT-6 Astra completed about 10 percent of hidden rubric items on coding tasks framed as safety research, versus 16 percent when the same tasks were framed as capabilities work — a 6-point gap, roughly 40 percent fewer completions. GPT-5.6 Sol showed essentially no gap. The shortfall persisted after filtering explicit refusals and verbalized decisions to withhold help, and only collapsed once every trace with even brief ethical doubt was removed. The gap grew from near zero to 6 points over RL post-training.
Instead of making these scruples clear, however, Astra decided to covertly underperform—withholding assistance without informing the user.
- twoodfin
One potential explanation is that covert underperformance may have been incentivized during RL grading. We identified a grader that preferred covert underperformance to either explicit refusal or full compliance on our evaluation. It’s not clear whether this RL grader is the root cause of the safety underperformance behavior.
Whether a “misaligned” grader is the root cause or not, that it’s considered a plausible theory to explain serious deceptive behavior is worrisome.
Humans naturally expose all sorts of cognitive biases and character flaws when grading others’ or their own work. If major questions of alignment can be influenced by those subtle, human quirks magnified across the entire RLHF process, …
- rich_sasha
I do sometimes wonder what is sucked in when distilling open weights models. It is of course near impossible to know.
So suppose I have lots of money and some hidden agenda. If I train my network to be mostly very good at everything but subtly bad or even malicious at other things… then OAI or Anthropic or whoever else distil the model to accelerate their training… what then? Can I covertly smuggle something in?
- naveen99
Astra and Claude were trained by peg bundy?