ByteDance's USDB cuts recommender write latency by 85.8%
USDB: Scaling Storage for Next-Generation Recommender Systems at ByteDance
Recommender models now chew through long, attribute-rich user histories, creating a storage asymmetry: many tiny continuous updates against repeated large reads. ByteDance's USDB unifies sequence storage for streaming and batch workloads using CascadeLSM time-based compaction, shared persistent files across differentiated replicas, partial dumps for inference reconstruction, and a hybrid columnar format for projected reads. Production migration shows 85.8% lower mean write latency and 42.1% lower infrastructure cost, while aggregation pushdown sustains 8.5x the throughput of ordinary reads.
Recommendation models consume increasingly long, attribute-rich user histories, making small continuous updates and repeated large reads a central storage asymmetry.
- bashhq_r
So bytedance has the luxury of building custom storage for this, meanwhile we're over here trying to shoehorn user sequences into redis sorted sets
- BenoitP
Setting up a classical DLRM at work, show of hands: who has managed to bridge the gap to generative recommenders or LRMs?
Our survey identified quite a substantial list of challenges: product cardinality (Semantic IDs?), low performance vs discriminative recommenders (our user intents are heavily biases towards repetitive products), cold starts, entity features, missing explainability suite, training cost, prediction cost, etc.
Has anyone done it easily? Where is the tutorial for generative recommenders ?
- mikert89
The future of recommenders is just an agent paging through your data and trying picking the best next options