ctrlb-decompose - Strip noise from logs for LLM-ready insights
Show HN: Ctrlb-decompose: Strip the noise from logs before sending to LLMs
ctrlb-decompose transforms millions of noisy log lines into a handful of actionable, structured patterns optimized for Large Language Models. By utilizing a two-stage pipeline featuring CLP encoding and Drain3 clustering, this Rust-based tool compresses raw logs while preserving critical statistics, anomalies, and variable correlations. Whether run as a CLI, a WASM module in the browser, or a library, it delivers token-efficient markdown or JSON output, enabling developers to feed clean, high-signal data directly into AI agents for rapid debugging and analysis.
Turn millions of noisy log lines into a handful of actionable patterns — with typed variables, quantile stats, anomaly flags, and severity scoring.
- hugopuybareau
Have you benchmarked it in real condition to test if it truly reduces costs ? I feel like many of those type of tools often end up making the llm more lost about the logs and make it more spend than just feeding the raw logs or letting it code to discover the content itself
- pranav_rastogi
Finally, a tool that tackles this problem without forcing you to adopt an entire ecosystem. I appreciate how this leans into the Unix philosophy by just focusing on the decomposition step and doing it well. Really appreciate the modular architecture here.
- firasd
Very interesting.
A few days ago I made something to use in addition to head/tail: Venetianblinds.js shows equally spaced chunks of a file https://github.com/firasd/venetianblinds
So I can kinda see these as being part of the same workflow like VBlinds for 'what does this file even look like' before calling the decompose