Reasoning Models Don't Solve an Objective—They Find Paths
What objective is reasoning solving?
Alex Wang reframes reasoning as pathfinding through token space: RL training maximizes the probability of landing on a correct answer across many sampled paths, but no single chain is guaranteed to converge. This view explains why forcing longer chains stops helping, why chains of thought aren't faithful, and why scaling RL can make reward hacking worse—the gains from "thinking longer" are really search paid for during training and amortized into the weights.
Reasoning isn't actually solving any objective at all. A path is not guaranteed to converge to the correct answer.
- htrp
>I'm a researcher based in San Francisco. I like to imagine where the world is headed, and what it takes to get there.
>I did my PhD in computer science at Stanford, where I designed long-context architectures for sequence modeling and memory. Before that, I studied math & physics at Cornell. Most recently, I was a research scientist at Cartesia, working on in-context tokenization and novel architectures (HNets).
Not the MSL Alex Wang
- derefr
It solves for the same thing a "train of thought" (internal vocalization) solves for in the human brain: there's a lot of stuff you "know" as episodic memories of words you've heard/read, but haven't yet internalized via practice into reflex reactions. It's a lot faster and cheaper to acquire a memory of some words, than it is to acquire an entire new skill, so you end up picking up far more memories-of-words than concrete skills from a given training corpus.
Reasoning (i.e. bubbling those remembered words up into your mental context, and then using a special "trigger effectively arbitrary synaptic activations dependent on the content of the attention-masked context and an ongoing state" bridging reflex on them, a.k.a. "system-2 thinking") is how you leverage memories-of-words into useful behavior. Rather than executing native skills embedded into your synaptic weights, you're running a single very fancy native skill that acts as an interpreter for word-memory data, slowly and strenuously turning words you're holding in mind into more words (synthesis), or into associations to other memories (word-memories or otherwise), or into actions/outputs, etc.
(An especially useful "trick" of this system is planning, i.e. chaining the interpreter through "mere words" that conjure more "mere words" to mind until you end up at words that form a declaration of what you're about to do. You can then rely on a special part of the bridging reflex, namely the "instruction-following" reflex, to […]
- piker
> Consider the problem: “what’s 1 +2 +3 +⋯ +10”? One solution is to add the numbers one at a time: 1, 3, 6, 10, and so on up to 55. Another way is to pair them from the outside in: 1 +10, 2 +9, 3 +8, 4 +7 and 5 +6 each make 11, and five pairs of 11 make 55. Both of these paths land on the correct answer, 55.
Watching my son work through difficult math problems has forced me into this realization. In my head there is one "correct" path, and I often have to stop myself from interrupting his chain of thought. He often is just getting there a different way.
- vova_hn2
I'm pretty sure that the current approach of baking in chain-of-thought reasoning into weights is not the end of the line. It just seems to accidental and ad-hoc.
P. S. The animated bird is truly magical!
- vatsachak
Alex also had the transformers as memory blog. Beautiful. Somebody give him the bag