Qwen 3.8 leaps toward GPT-5.5 Pro reasoning with a tiny prefill boost
Qwen 3.8 follows GPT-5.5 Pro reasoning prefills
A v1.1 rerun of the reasoning-prefill experiment uses GPT-5.5 Pro as the teacher. For each of 45 problems, the author generated two responses from each target model: one ordinary, and one starting with the first 1% of GPT-5.5 Pro's reasoning. Measuring unigram source recall in the first 100 tokens, Qwen3.8 A95B jumped +20.58 percentage points (from 33.92% to 54.50%), with the largest gains on STEM and synthetic puzzles. Kimi K3 also showed high overlap but a smaller prefill effect (+4.31 pp). The data hint that Qwen may have learned from GPT-5.5 Pro or a close relative, not from Opus.
The data suggest that Qwen may have learned from GPT-5.5 Pro, or from a closely related GPT model, rather than from Opus.
- wongarsu
That writing style might be a tad too tense
If I got it correct (appending B from https://stolen-thoughts.com/paper.pdf is essential) they are the authors of the well-known exploit to recover readable CoT from OpenAI and Anthropic models. They use that to find hints of distillation, by running a benchmark with a SotA model, recovering the CoT, then taking the first 1% of the CoT and running the open-source model as if that was the start of its own CoT. In the paper they found that Kimi-K3 gets a lot closer to Claude 4.8 answers when prefilled with the start of Claude 4.8 reasoning, suggesting that Claude 4.8 was used in its post-training. This blog post is the follow-up with results that suggest that Qwen3.8 was post-trained with the help of GPT-5.5 Pro (or some similarly responding GPT model, it's unclear how many models they tested)
- nzeid
I see comments that this overlap between Qwen and GPT is due to rogue training or post hoc training. Did it occur to anyone that maybe the two sets of models were trained directly on the same solutions to the researchers' benchmark?
- c7b
I wasn't aware that we have access to raw reasoning tokens? I thought what you get is a kind of summary. Does the author have some kind of privileged access or was my assumption wrong?
But for the question studied here it probably doesn't matter - overlaps in the publicly available output may be indicative of distillation (or not), regardless of what it is. I would just find it surprising that the Chinese labs would use it so trustingly. The publicly released reasoning trace is the first place where I would suspect some distillation poisoning to be injected.
- 7734128
The problem with this is obviously that the only GPT 5.5 thoughts that we have access to are from stolen thought.
Qwen 3.8 0902 was trained after the release of the paper on August 10, so it should have seen those specific thoughts.
- hermitShell
As a user of local models, does this mean that there are 'magic incantations' that can increase the performance of some local models?
I see some details about recovering information via whatever technique. It's interesting, but appears not generalized.
So for a specific question, yes, but this is not about techniques like adding a good embedding that just generally tends to improve open model performance on certain tasks.