Probing GPT-5 and Opus reveals their hidden training timelines
Exploring Claude/GPT Knowledge Cutoffs and Pre-Training Timelines
By quizzing frontier models on daily Wikipedia facts, I estimated the knowledge cutoffs of GPT-5.6, Opus 4.7+, and others. The results suggest Anthropic's Opus 4.7 through Opus 5 all share the same pre-training cutoff, while OpenAI's GPT-5.6 family comes from a separate, later checkpoint. I also asked models what date they think it is and what they think they're called, revealing that some Anthropic models self-identify as GPT-4 and that vertical bands in the data hint at training on past model outputs.
It’s a bit spicy that OpenAI models never identify as another lab’s model (besides briefly a Tesla Model S) yet Anthropic’s Sonnet 5 will regularly self-identify as GPT-4.
- tobwen
From my own experiments with various models, I suspect that LLMs have distinct/partitioned cutoff dates; for example, historical literature doesn't change (Greek history, Shakespeare, Goethe), general knowledge (updated only in certain areas), technologies (updated regularly), software also remains surprisingly stable - for example, with GIT, a basic command set is sufficient to do 99% of the jobs - new features are unknown or unnecessary, and tabloid knowledge, which is always up-to-date (politics, Taylor Swift albums).
- myworkaccount2
I wonder if this kind of analysis will give us a way to check if the frontier labs are waiting for the right moment to release their models.
To me it feels obvious that these companies are not releasing models as soon as they are done doing their post training / testing with any new model.
But there is no real way to know how much of this "waiting" any lab is doing, if we can get better estimates this way maybe we can gauge how far the open weights models really are.
- rad-b
Great read and interesting analysis! I’m less charitable toward Anthropic supposedly not distilling ChatGPT for training purposes. Maybe not today, but during the GPT-4 era when Anthropic was the underdog - I can see it happen. Packaged along with some of Amodei’s clever jumping through hoops to prove how that is, in fact, virtuous.