celld's simulator uncovers hidden alarm race in Cloudflare Workers runtime

Deterministic Simulation Testing in Celld

celld, a runtime for Cloudflare Workers and Durable Objects, uses deterministic simulation testing to reproduce bugs that depend on event ordering. The simulator controls task scheduling, storage responses, and simulated time, using seeds to replay failures. It found an alarm race where a delayed cleanup deleted a wake entry needed by a new alarm, causing missed alarms after a node restart. The team fixed the design and added a regression test.