Context Language Models manage their own context as a file, beating SOTA with 21.5% fewer FLOPs

Researchers introduce Context Language Models (CLMs), which treat context as a file the model can update freely, learning what to keep. Built zero-shot with existing models, CLMs outperform state-of-the-art context management: 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus, 5% higher scores with 59% fewer FLOPs on 12-hour EdgeBench, and 65% greater improvement on a 24-hour multi-repository agent-swarm task. Natural-language instructions and online RL further boost performance, while Suffix Cache Reuse cuts serving compute by 35%.