GitHub's code search folds case at 45 GiB/s by removing an optimization
Don't stop early: Case-folding source code at memory speed
GitHub's Blackbird code search engine processes over 480TB of source code, requiring case folding for every byte. A seemingly counterintuitive optimization—removing an early-exit branch—enabled vectorization, boosting ASCII folding from 3 GiB/s to over 45 GiB/s. The team open-sourced the Rust crate 'casefold', which also handles Unicode folds efficiently with a compact 1776-byte table, avoiding heap allocations and decoding only when necessary.
The early-exit is what gates vectorization: keep the break but make the body perfectly branch-free and you still get zero vector instructions (~2.6 GiB/s); a data-dependent loop exit is enough on its own to keep the loop scalar.