LLMs Beat Human Records on Zachtronics' TIS-100 Programming Puzzles
LLMs Play Zachtronics' TIS-100
Sixteen LLMs were given TIS-100 and its manual but no access to the community leaderboard, then told to keep optimizing. They produced 133 new Pareto-frontier points across 18 of 55 puzzles, including three outright cycle records — Opus 5.5 cut one tournament puzzle from 2,694 to 1,918 cycles. Opus 5.5 and GPT-6 Astra lead and can't be statistically separated, and cost doesn't track quality: GPT-6.1 Sol reached 75.6 record proximity at roughly $0.75 per puzzle versus Opus 5.5's $5.08.
Within days, players picked up the models' programs and improved on most of them.
- xpct
Anyone have any ideas for how we can still have Zachtronics-likes with a human leaderboard in the future, or is that something that just won't be possible?
- rpdillon
TIS-100 was a mind-expanding game for me. I kind of wish I'd finished it (I think I made it ~20 or so in). It always felt like the sort of game that should be ported to the web or to a TUI, even if just as a sandbox (not bringing over the specific puzzles). Should probably revisit it!
- xnorswap
One of my all-time favourite games. I almost completed it, but got stuck on one level I could never quite wrap my head around.
I don't want to spoil myself looking at solutions nor get help with LLMs, so every couple of years I try to revisit it, but it's difficult to dive straight back into a hard puzzle.