Rebuilt AI comment detector hits 77% balanced accuracy on public data

Better AI code comment detector

The author rebuilt their AI comment classifier on a fully public dataset, fixing the shaky foundation that prevented sharing the previous version. The new tool, available as a web app that runs entirely in the browser, achieves a balanced accuracy of 77% on synthetic data and 88% on a small real-world test set. It also outputs calibrated confidence scores, so users know when to trust its verdict. The post details the data collection pitfalls—like subject leakage and failing to strip docstrings—and explains the feature evaluation process that guides the classifier's decisions.