OpenAI cuts reasoning-extraction attack success from 54% to 0.2%
Combatting Adversarial Distillation
OpenAI details how it fights adversarial distillation, where bad actors train models on a frontier system's outputs to inherit its capabilities without its safeguards. Its defenses span adversarial training, classifiers, and API controls, plus investigations into coordinated extraction campaigns. In adaptive red-teaming, the average success rate for recovering protected reasoning fell from 54.0% on GPT-6 Astra to 0.2% on GPT-6.2 Astra. OpenAI also flags national security stakes, citing DeepSeek and Moonshot AI.
The average attack success rate fell from 54.0% against GPT‑6 Astra with its original safeguards to 0.2% against GPT‑6.2 Astra with its updated safeguards.
- WiSaGaN
It's just sad this company is named OpenAI, and then put out things like this. You trained all your models without giving back to authors, coders, directors, mathematicians . And you are putting them out of work. And finally you want to make sure only you have such models. You people won't go down well in human history.
- aenis
I'd expect that if those techniques are successful, it will have the same effect as putting your valuables in a safe in a high crime neighborhood. Before you know it, somebody puts a gun to your head and ask for the keys to the safe. Its only a matter of time before state sponsored groups exfiltrate weights of some leading model in entirely criminal ways. One would hope that those companies are well aware of that, but criminals only need to succeed once, and the incentives are in the billions.
- bob1029
Distillation comes in many forms.
Hypothetically I can put a black box around their black box and log every single token in and out as part of ordinary business. This can be a profound amount of data. I can turn around and train my own model using this data. How would anyone detect this? Reasoning tokens are only so much of a moat.
- pacificmaelstrm
"Life finds a way"-Jurassic Park
- samrus
Hypocrites