Every Fast Write Moves Work Somewhere Else
A deep dive into storage engine write paths reveals that the fastest acknowledgment points—like copying to memory or syncing a local SSD—shift durability risks elsewhere. The article contrasts local WALs, network volumes, and object storage, showing how batching flushes improves throughput but complicates failure semantics. It argues that unlabeled latency numbers are misleading without knowing what failures the acknowledged write survives, and that every speed gain comes with hidden costs in durability or cleanup.
A local fdatasync() measured at 1 ms would make the local path about 6x faster, while 0.1 ms would make it about 64x faster.
- aleksiy123
Today I learned there’s a name for the general version of this idea.
https://en.wikipedia.org/wiki/Waterbed_theory
At a certain point in a solution everything you do to optimize (“push”) in one area will cause a negative effect in a different area (“bulge”).
But this is a nice concrete example.
- cpard
When I saw the title the first thing I thought of was schema on read versus schema on right when in data platforms.
You can make writes faster and part of that is by not dealing with schema resolution but you do push the work somewhere else there too.
I guess the same principles apply on many different levels, from when you write to the file system up to how you deal with conflicted data types during data ingestion.
- xixixao
2 angles I think DB designers don’t often think about:
1. Durability extends to the client. Replicated db might ack a write to client, but what if that ack gets lost on the way back over network? If client talks to the DB over simple HTTP, the write might first look like a failure. Can the client retry?
2. Human perception times are biological and don’t change much. But everything in the tech stack has gotten so so much faster since the 80s. Throughput matters, sure, but latency (relatively speaking), is much less of a constraint now than it was.