If Scrubs Hurt, Your ZFS Design Is Broken
When your monthly ZFS scrub cripples production performance, it signals a fundamental design flaw rather than a scheduling issue. I explain how oversizing pools for capacity while ignoring IOPS leads to fragmentation and latency. The solution involves offloading metadata to a dedicated special VDEV using enterprise SSDs, ensuring maintenance tasks like scrubs and resilvers run smoothly without disrupting your application workloads.
The scrub did not create that condition. It measured it.
- zenoprax
I have attended some of the webinars and listen to Allan on a podcast regularly and this LLM-written post is painful to read. If I hadn't already been familiar with him and his work I would have not given much weight to what was written. He is a big proponent of tuning ZFS which requires knowing your workload for which he has written other guides.
1. From my own experience, most people who use a ZIL for faster sync writes are wasting space. You only need five seconds of peak throughput (for a 10Gb connection that's ~5 GB). Create a pair of 25 GB partitions for the ZIL rather than dedicating entire devices.
2. With the remaining space set up a mirrored Special VDEV with heavy use of small blocks. 32k block sizes will still give some room for compression which, other than saving space, can increase throughput and reduce IO/ps.
3. Pause scrubs (`zpool scrub -p`) when the workload is either expected to be heavy or is actually heavy based on moving averages and resume when its safe. Even my 24 TB NAS takes 24+ hours with no other activity.
- toast0
> Sizing is measurable rather than guesswork. zdb -bb tank breaks down allocated space by block type; sum the non-data categories and you have your current metadata footprint.
Of course, this is also not a fast command on a slow array; my half full 2x 24TB mirror is estimating ~ 12 hours to run this; the last scrub took just over 24 hours to run. (After I posted, the remaining time dropped to 10 minutes... not as slow as it seemed, but still slow, and using a lot of I/O ... left for lunch and 50 minutes later it's almost done; estimation is hard)
- assimpleaspossi
ZFS scrub performance shouldn’t come at the expense of production worklaods. If scrubs cause latency or downtime, your pool lacks the IOPs and metadata performance needed for healthy maintanance.