Frame selection is the whole game: notes on making LLMs watch video

Feeding a video to an LLM instead of a written summary lets the model see the raw event, not a human's edited version. But token budgets force tough choices: you can only keep ~100–150 frames per video. The author's open-source tool uses adaptive scene detection, three-channel deduplication (global, action, and settled), and fuses frames with transcripts to make every frame earn its slot. It's served via MCP and runs locally, MIT-licensed.