Stop Calling LLMs Next-Token Predictors

Stop Thinking of LLMs as Next-Token Predictors

The common description of large language models as 'next-token predictors' is technically accurate but misleading. While LLMs do emit tokens one by one, modern post-training techniques like reinforcement learning with verifiable rewards (RLVR) enable them to learn from novel sequences generated through exploration, not just from existing text. Using a chess analogy, the author distinguishes between a system that predicts the next move in a database of grandmaster games and one that chooses moves to maximize winning probability. This shift in learning objective means LLMs can encode knowledge and behaviors never seen in their training data, making the next-token framing an incomplete mental model.