Researchers Steal Hidden Reasoning from Anthropic, OpenAI, and Google LLMs via Replay Attack
Stealing Reasoning Traces from Proprietary LLM APIs
A new paper reveals that proprietary LLM APIs from Anthropic, OpenAI, and Google return encrypted chain-of-thought blocks that can be replayed across sessions, users, and models. By injecting a trace from a frontier model into a weaker, jailbroken sibling, the researchers recover the stronger model's hidden reasoning in plaintext, without attacking the stronger model directly or triggering anti-distillation safeguards. The attack works on claude-opus-4-8, GPT-4, and Gemini models.
Proprietary reasoning can be recovered from its encrypted traces.
- Groxx
>We take a trace produced by a frontier model, replay it into a weaker sibling, jailbreak the weaker model, ...
Ha! I've been wondering if replaying across models would work, ever since https://blog.cryptographyengineering.com/2026/05/29/fooling-...
I'm honestly rather curious if this was intentionally allowed, it's the sort of validation that's easy to miss (particularly if you're wading into the vibe waters). Seems like something that'd be absolutely riddled with possibilities for shenanigans.
- andai
If I'm reading this right, they literally just ask a LLM to tell them what the traces say, with the key being that the traces are portable across LLM models, so they can switch to a smaller one that's easier to jailbreak.
- vhantz
> For some AIME problems Opus 4.8 sometimes states the answer before deriving it. We find that the API summary does not always preserve this distinction, and can instead make the reasoning appear like a clean derivation.
No surprise here but good to have more confirmation that they just put all that in the training data. And based on the "reasoning", the models have some form of index of those problems (or they are HEAVILY trained on them).
- niemandhier
You cannot steal what is not owned.
At least in the EU there is no copyright for LLM outputs, so I guess all they might do is violate the terms of service.
- SwellJoe
"Stealing" is a strong word to use for looking at the words produced by models built from the collective commons of the world.
And, honestly, being able to see how LLMs make decisions is critical to trust and security. I consider it a valuable feature, somewhat akin to seeing the source of software I use.
- Pragmata
Apparently you can do the same by simply running it without reasoning, while giving it a thinking tool...
>guys you do know you can just disable thinking, and instead give it a "deep_think" tool, and it will call it with internal CoT reasoning format right?
>gl fixing that
- sly010
"Recovery" would be a more apt (although less catchy name). The stealing is on the provider side for not giving you access to tokens you already paid for.
- Aissen
"Stealing" something you already paid for (tokens), but that you can't have access to(!). And trained on the sum of human knowledge.
Training on other model outputs ought to be business as usual, stop using morally charged terms made up by future monopolists: https://thomasdullien.github.io/posts/2026-06-15-rl-economic...
- glub
I did this with Codex's recent encryption of compaction.
Interestingly, I didn't have to drop to a dumber model, just a 2 sentence <developer> prompt auto-injected before and after compaction made all their models output the encrypted compaction data in plaintext.
The result was... interesting. There's nothing unique in there and I still don't understand why they decided to encrypt it in the first place.
- myworkaccount2
Is this how the eastern labs "distill" SOTA models?
If you can play it right, you don't even need to send suspicious prompts to the frontier models. Just use them for regular tasks, extract the encrypted COT blocks and replay it to a cheaper model to get the plain text COT.
But the real question is: Is it okay to steal from a thief's hoard?