Cloudflare halves DNS build delay and cuts memory use by 85%
We halved DNS build delay and cut memory use by 85%
Cloudflare's Zone Builder now handles about 700 DNS record changes per second, and some zones hold over 20 million records. A new targeted build planner works out the smallest build it can prove is safe, so a single zone setting change no longer forces a rebuild of the entire zone. A unified scheduler merges work for the same zone. Peak memory fell 85%, and the freed headroom let Cloudflare shorten the pipeline's built-in waits, halving the time from a DNS change to its data reaching Quicksilver.
The rule is simple: every key a targeted build writes must be byte-for-byte identical to what a full build would write. If we cannot prove that, we do a full build.
- thephyber
I have been wondering if the memory scarcity would lead to a desire to optimize software to be more efficient.
Curious what the R&D / efficiency development ROI is on reducing memory footprint. I suppose we should both consider current ROI and near future ROI (as the price of memory will likely stay high for at least another 1-2 years).
- mhovd
Cloudflare doesn't get enough credit for their series of overall great technical blog posts.
- InsidiousWombat
Isn't 700 changes a second like... really low, considering their scale and amount of domains they manage ?
- JulianPaul
I'm very curious about the bugs (before vs. after relational) that come from an improvement like this over time
- likoya
I used to use cloudflare dns, but the same cloudflare harassed me the entire day testing if I'm a robot. The harassment ended after I switched the dns server. See you never again, clouflare