Why Better Data Beats Better Models: Linum's Data Filtering Journey

Getting video models to learn better, faster

Linum's field notes detail how they improved generative video pre-training through data filtering, not model architecture. They trace their evolution from 2024's CPU-based CV heuristics (PySceneDetect, EAST, H.264 motion vectors) to 2025's finetuned LLMs and reinforcement learning, emphasizing the importance of removing noisy data, rebalancing categories, and filtering out un-captionable clips. The post argues that most gains in video models come from data improvements, not architectural changes.