Tiny 1.8M-parameter model solves Sudoku-Extreme with 345× less compute
Energy-Based Reasoners: 385x more efficient inference

Researchers at Duke, UIUC, and Harvard introduce Energy-Based Reasoners, which learn an energy function over latent reasoning states and treat looped updates as gradient descent on that energy. The energy scores candidate answers, its gradient guides refinement, and a small gradient signals when to stop. With just 1.8M parameters, EBRs hit 99.9% on Sudoku-Extreme using 345× less inference compute than the next-best model, and they set state-of-the-art coverage on N-Queens and Graph Coloring.
Correct trajectories settle at low energy with small gradients, while wrong ones keep moving.
- Topfi
Recommend checking out Logical Intelligence, chaired by among others Yann LeCun, for more context on EBMs [0]. Has great potential in areas that LLMs traditionally do not perform that well in and, because they want to show the advantages of EBMs, also tends to be a great source of examples LLMs struggle with, like primary school level maths and counting (while ironically providing Millennium level Lean proofs). Does of course open the not unreasonable discussion of whether it matters if LLMs can accurately count when writing code to do the calculation is reliable and fast...
[0] https://logicalintelligence.com/blog/energy-based-model-sudo...
- program_whiz
A clarifying way to think about this for sudoku:
For every clue, mark the probability of sampling its number in the row/col to zero. Also mark the probability of selecting a "clue cell" for modification to 0.
Pick a random cell for modification, based on its "energy" (higher energy get modified first):
1. Every conflicting cell gets energy +1.0 (try to resolve conflicts before new empty cells). E.g. two 7s on the same row both get +1 energy.
2. Empty cells have small energy 0.1
3. Filled cells with no conflicts have energy 0
For the selected cell, choose a digit. Look at numbers in its row/column, every occurrence of a number in the row/col drop prob of sampling by 90%. Example:
| No. of 7s | chance of 7 |
| --- | --- |
| 0 | 10% (equal to other digits) |
| 1 | 1% |
| 2 | 0.1% |
Obviously this would get more complex with more numbers on the line (this is just checking 7 on otherwise empty line).
This is essentially what these models are doing. It provides a kind of "wave function collapse" search with random exploration based on "energy", but energy is just defined as conflict / unsolved. For something like a maze, energy is disconnected path segments or walls, etc.
- mocmoc
I've spent a year on this. EBMs while ... how to say it, they are impressive validators , it's insanely hard to train them. They are very niche specific. While LLMs , being not right all the time and clumsy and so on, they help the people to get the job done at the end.
EBM sudoku is not that hard in a small scale, and Logical Intelligence sudoku solver doesn't scale , if I'm wrong , I want to see them offering as big of a sudoku as you want.
also, after months and months training EBMs , you come to realize... it's really worth it?
- xgk
The paper does not seem to define what Energy-Based means. The paper says: "EBMs learn a scalar function that assigns an energy to a candidate prediction given an input." What codomain do those learned scalar function have? I guess real numbers? But is "Energy" just a synonym for the reals?
- igleria
As far as I could understand, this works very well for a very specific logic puzzle problem set. Interested on future papers tho :)