Schema Harness Achieves 99% on ARC-AGI-3 by Thinking Like a Physicist
Schema Harness Achieves ~99% on Arc‑AGI‑3 Public
We introduce Schema, a new harness that pushes frontier models like Claude Opus 4.8 and GPT-5.6 Sol to achieve 99% on the ARC-AGI-3 Public set. Instead of changing model weights, Schema changes the process, forcing agents to jointly solve state grounding and mechanism discovery. By treating the world representation as an editable program, our system allows agents to revise their understanding of objects and rules when predictions fail, mirroring how physicists refine theories.
When predictions fail persistently, they do not only adjust the law. They change what the state is.
- vessenes
To be clear, we’ll want to see how this performs against the hold-out set. If it holds up, though, it’s a big deal, and kind of in line with the vibes this year, which I’d typify as ‘harness matters’. Maybe we’d upgrade to ‘harness matters immensely’ if this can 100% ARC-AGI-3 on existing models (more in the 13% range without this harness).
I’m pretty excited to see what sort of generalization we come to over the next 12 months on the harness side: if it turns out this can be RLed in as ‘consider if building a world model might help here’ and we get this as another native capacity, that will be interesting. If we get 100 of those problem-solving strategies all included, feels like we will see another hurdle cleared in terms of usefulness.
- ClassAndBurn
Any custom harness for a problem shows that harness engineering is going away. Eventually models will introspect problems, then build custom harnesses tailored to that. Then use and modify the ephemeral harness as required.
Sol Ultra style is the path forward. The models are smart enough to self serve their tooling and processes. Given a problem they can figure it out and ask for directions when needed.
- stared
In the spirit of ARC-AGI-3-like challenges, we just tested if frontier AI models are able to solve a lovely puzzle game, Baba Is You: https://quesma.com/blog/baba-is-bench/
A year ago, Sonnet 4 barely solved the first level. Now, both Fable 5 and GPT-5.6 Sol beat the first two stages. GPT 5.2 is slow, but efficient, while Gemini 3.1 Pro and 3.5 Flash struggle.
- teravor
it looks like what they are doing is using a frontier model to write a simulator for a game and then solve using it.
it's not as impressive as it looks. the goals of Arc-AGI-like constructs is to get an IQ-like figure using raw'ish 2D measurement 'games' in the hope that it would signify something meaningful.
what this harness does is get the model to write a simulator first, it's measuring something entirely different.
- gandalfgeek
Big jump for sure, but definitely comes with a giant grain of salt lacking open-sourcing the harness itself and measuring performance on the held-out set.