NEEDLE: The live benchmark your search engine can't memorize

Needle: The benchmark your search engine can't memorize

Static benchmarks let search engines cheat—by memorizing answers or even downloading them from HuggingFace mid-test. Keenable's new live benchmark, NEEDLE, refreshes tasks hourly and daily across news, finance, scholar, rare-entity, and legal queries, making overfitting impossible. Early results reveal that engines with independent indexes, like Keenable itself, outperform those that merely federate others, and that traditional engines optimized for human clicks fail at agentic search. The benchmark tracks improvement over time, showing who's learning fastest.