Durable execution without history replay: checkpointing the program continuation
Most durable execution systems recover by replaying retained history to reconstruct a program's position. The author argues this makes accumulated history part of the recovery path, which becomes costly for long-running agents. He proposes Transparent Continuation Checkpointing (TCC), which captures and restores the live continuation instead. A controlled test with ~4 KB live state and durable-boundary depth from 10 to 1,000 showed TCC recovery stayed between 0.6 and 0.9 ms, while Temporal replay reconstruction grew from ~61 ms to 1.7 s.
A program that has performed ten thousand operations but retains a small live continuation should not necessarily become harder to recover simply because its past is long.
- weitendorf
Good model. Anybody interested in actually training models or designing agentic systems should be doing this.
My company started around working on this problem because it's the basis for how you train programming models/reliably deploy LLMs to do specific tasks. It allowed me to build a much better mental model for LLMs because I saw how weirdly fickle/inconsistent/picky they could actually be outside of a "chat" where it feels like they have a coherent persona or consistent knowledge/capability.
Initially I thought of it as a search over prompts for capability at completing specific tasks, but now I think the speed/reliability and operations (eg can I switch models without degrading perforamnce?) benefits are even bigger benefits for most users.
A little "secret" since labs are making it harder to even use their models in this way and it's important that it be more widely understood: distribution-aware replay/re-sampling is a key technique in post-training LLMs. But it's also something that allows you to automatically identify the best model for some subset of your tasks, which can save you a lot of money.
- orbital-decay
Durable execution looks a bit like a buzzword in general. If your state is defined in the execution graph then it's just an umbrella term for a group of pre-existing algorithms and patterns. If it's undefined then what are you resuming to? The snapshot just before the crash likely leads to the undefined state again, in which case you're durably automating the crash (or even worse, uncaught incorrect behavior).
- alex_hirner
Snapshotting programs is also what https://github.com/pydantic/monty enables and aims for.
FWIW, I think we'll see a rise of AI-ready interpreters. In some sense, I like that it challenges traditional microservice architectures as an aside.