IPinfo shrinks its MMDB databases by a third without touching a byte of data
How we made our MMDB files a 33% smaller
IPinfo found that MMDB writers store duplicate subtrees in the search tree, so it applied hash-consing — a technique from compiler research — to share identical nodes. Across 89 daily databases the catalogue dropped from 91.2 GB to 62.8 GB, with ipinfo_privacy falling 80%. Files stay standard MMDB, so every reader works unchanged, lookups return identical results, and memory use falls too. The tool is now open source.
In our files, a large share of the tree was duplicate subtrees.