Goodhart's Law Comes for Every Benchmark You Trust
The article explores how Goodhart's Law—'when a measure becomes a target, it ceases to be a good measure'—applies to modern AI benchmarks. As benchmarks like MMLU and HumanEval become targets for optimization, they lose their ability to truly measure capability. The author discusses the arms race between benchmark creators and AI models, the rise of benchmark overfitting, and the need for more robust evaluation methods that resist gaming.
When a measure becomes a target, it ceases to be a good measure.
- astro1234
I agree and that’s why we need and indeed have an ever evolving landscape of benchmarks
> Private, refreshed test sets attack the mechanism itself, and in my view they are the only intervention that does. If the questions have never touched the public Web, they can’t be in the training data; if they rotate, memorizing this year’s set doesn’t help next year.
That’s what we have. A fresh public benchmark is also good, and teams do make efforts to decontaminate training data but there’s likely just no great way around leakage.
Btw, lots more issues in benchmarks than the ones discussed; for instance you can leak answers from the questions themselves or in the case of e.g. multiple choice formats in the actual answers. You just pass the MCQ choices themselves to the model and it may be able to guess way above chance. Coding agent benchmarks sometimes forget to delete .git. They mention e.g. a 6.9% error rate in one of the benchmark items, this seems pretty typical and I would actually be fine shipping that.
Benchmarks are very ugly, but if they didn’t exist we would need to invent them. All of the problems above and more do not explain the progress we see. There are probably 50,000 benchmarks in the literature and new ones get created frequently with varying levels of quality and usefulness.
- cyanydeez
obviously, the best benchmark is the one you tell no one about.
- teddyh
“When you place a tangible value on trust, trust becomes a commodity to be bought and sold.”