How Antithesis Found a Decade-Old SQLite Bug in 15 Minutes
Breaking the WAL
Carl from Antithesis recounts how he used his company's tool to discover and verify a long-standing SQLite WAL-Reset bug in just 15 minutes, while on a road trip. The bug, which had lurked since 2010, caused Tailscale six months of uptime issues. With generic assertions and a simple workload, Antithesis pinpointed the issue, and the fix was confirmed in a subsequent run.
By giving agents the skills to use Antithesis, I’d just found and verified it in like an hour, from my phone, sitting under spruce trees in the sunshine.
- carlsverre
Hi! I'm the author and the person who was on a road trip when I decided to do this experiment back in July. I'm super excited to finally get the blog post out, and even better, alongside a post describing the root cause process that went into the project. You can read about Tailscale's story here: https://tailscale.com/blog/sqlite-wal-reset-bug
I'm only sad that I didn't put SQLite under test earlier in the year, or I would have found this issue right away. If you look at the workload[1], you can see how simple it is. Exactly the same kind of workload we write every day to help our customers find bugs.
If you have any questions about our process or how debugging with Antithesis works, please let me know! Thanks for reading!
[1]: https://github.com/antithesishq/sqlite/blob/3.51.2-instrumen...
- minimaltom
I went clicking through to see if I could find the prompt they fed the AI to locate the issue / write the test suite.
I couldn't find it, so its unclear if the prompt was completely "make a test suite" or was lead towards finding it in the first place, which wouldn't be a fair test.
The closest I mention of the prompt I could find was:
> Then I asked it to write a simple workload which exercised the WAL insert and checkpoint code. Notably, this is a completely generic workload.
With a skeptical lens, unclear.
- wwilson
We wanted to publish this ASAP since people are talking about this bug today. In a follow-up, we will show how our automated causality analysis could have saved Tailscale and SQLite from 6 months of root cause analysis. Stay tuned!
- uhohherewegoaga
as a long time lurker who usually enjoys antithesis posts, this was a pretty existentially sad read. we all know that company blog posts here always tend to be ads at their core, but historically afaicr antithesis generally toes the line well of keeping the "thing we sell is really good" as a side dish to the "i (author) did very interesting deep thinking and/or hard work to accomplish something cool" entree.
this one falls very flat on that front imo. the "antithesis finds rare bugs" is the overwhelming majority of the actual content here, where as the usual meat of the blog, the hard work, the deep thinking, really distills down to "typed this into claude mobile, typed that into claude mobile". yes, its a statement to the power of the technologies of today that's all it takes to find this bug, but man it's a sad signal for those of us looking for compelling technical reads instead of b2b sales pitches.
- john_strinlai
related: Tailscale Traces Database Corruption to 16y/o SQLite WAL-Reset Bug (tailscale.com) https://news.ycombinator.com/item?id=49272832