Most lookups ask about files that don't exist — a Bloom filter cut this Cassandra cluster by 60%

Most lookups are for things that don't exist

A service answering one question — have we seen this file hash? — was hammering Cassandra with millions of requests a second, and about 80% of them came back "not found." Caching misses was useless, since most misses are hashes never seen again. Putting a Redis-backed Bloom filter in front of the cluster let the filter answer most of those negatives outright, shrinking the cluster by 60%. The hard part wasn't the filter; it was convincing skeptical teammates it couldn't return a wrong answer.