Over 30% of New arXiv Submissions Now Read as AI-Written
Over 30% of new ArXiv submissions now read as AI-written
We analyzed 12,750 arXiv papers and found that roughly one-third of recent submissions read as machine-written. By calibrating our detector against pre-ChatGPT texts to ensure a minimal false-positive rate, we observed a sharp rise in AI-like writing, particularly in Computer Science and Quantitative Biology. While our results provide a lower bound on adoption, they highlight a significant shift in academic publishing since the arrival of large language models.
"If a tool marks 40% of new papers as machine-written but also marks 20% of papers written before ChatGPT existed, the real story is the 20% nobody mentioned."
HN discussion
- One user demonstrates that Pangram detectors can be trivially fooled by humanizing Opus 4.8 output with Grammarly, resulting in a false 100% human-written classification.
- A Stack Overflow contributor reports that their own human-written answers were flagged as 85-96% machine-generated, highlighting the danger of applying classifiers outside their specific training domain.
- Critics argue that AI detection is inherently flawed because the outputs of detection systems can be used to train better generators, leading to an equilibrium of digital uncertainty where only utility matters.
- While some detectors like Pangram claim low false-positive rates, others note that well-crafted human writing is increasingly dismissed as AI when it challenges readers intellectually or politically.
- Practitioners in basic biomedical research observe that LLMs are largely ignored because they cannot parse raw wet lab data, distinguishing them from machine learning tools used in other domains.