AI-Written Referee Reports Threaten to Collapse Peer Review, Economists Warn
Can Peer Review Survive AI-Generated Text? [pdf]
Large language models are now writing papers and referee reports in economics, and two economists argue that without shared norms, peer review could collapse. Their experiment with 25 real reports from Nature Communications found that AI detectors flag reports written entirely by AI, but not those merely proofread or edited by it. They propose detectors as a standard tool, closer scrutiny of flagged papers, and treating AI-generated referee reports like plagiarism, since editors could produce the same output themselves.
The real problem is not that the papers and reports were AI-generated; it is that no one had done the thinking behind them, and it showed.
- marginalia_nu
To be fair the peer review model has been falling apart long before AI generated text was a thing, mostly due to the exploitative nature of academic journals leading to incentive structures that don't align with producing good science.
Reviewers don't get paid and have zero incentive to do a good job. You can in many cases just submit a paper over and over to different journals until it encounters a reviewer that can't be bothered reading it, upon which it gets rubber stamped through. The result is that time and time again, incredibly shoddy and too often downright fraudulent science has gotten through peer review.
- patrickmay
Abstract, for those not interested in reading TFA:
"Large language models now write papers and referee reports in economics. The field must develop best practices for handling AI-generated text in the publication process. Otherwise, peer review is in serious danger of collapsing. We propose that AI detectors should become a standard tool of the profession. Our experiment shows that they flag referee reports written by AI, not reports that AI merely proofread or helped edit.
Aided by a simple model, we argue that a paper flagged as AI-generated should trigger a closer reading and can be rejected at the first mistake, and that an AI-generated referee report adds nothing that editors could not get by running the prompt themselves. Such reports are unacceptable: they damage authors and journals and should be treated like plagiarism. Individual restraint has already shown its limits, and the profession needs to develop norms to solve this collective action problem. We close with a practical guide for authors, referees, and editors."
- FabCH
I am an occasional reviewer.
Peer review has been dying for a long time.
IMHO the solution was simple before AI and it is simple with AI:
Double-blind review, but after the submission process both the article and the reviews are made public, with full attribution.
- astro1234
I’m going to say a class of opinion that has historically been overwhelmingly rejected by this crowd:
Peer review will no longer matter in about 2 years and it will feel silly that we were ever asking ourselves these questions.
Despite being proven time and time again that performance trends are incredibly stable and if anything accelerating slightly, that pretraining scaling laws and their analogs for post training are reliable enough for this to be mathematically uncontroversial, it is just as reliable that these clearly accurate projections are so incredible that people still don’t seem to accept it. This sentence has been true and could have been said at any point in time over the last 4-5 years (and it has, many times).
The AI 2027 scenario while hilarious when posted is alarmingly not that far off from where we are today. As Dwarkesh reports from his conversations with Anthropic researchers and those at other labs, we are on track to solve RSI sometime in 2027 (again a relatively easy thing to project if you know a little bit about performance trends and how model performance is measured).
The world will not implode like a dying star, and I expect the experience to be more or less what we’re going through today, though some capabilities will experience a much higher rate of improvement, and since there are unavoidable bottlenecks that AI cannot get around ever and some that will take years to figure out how to break down. But “oh no we don’t have humans to peer review” […]
- imeanwhatdoikno
Am not in academia, so the following might be naive, but it's not plausible that we go back to days before LLM writing and LLM reviewing papers. The amount of LLM output is also unlikely to ever decrease (and from what I understand the number of papers written even in pre-LLM days was already overwhelming the system). So:
1. Papers could be self-published on the web (Arxiv or Github or wherever else).
2a. Papers could be announced/advertised primarily by informal means (X and other social media, word of mouth, etc)
2b. In addition, there could be formal periodic announcements by academic institutions or industry entities that the authors belong to. E.g. a university might publish a list of top 10 (for your preferred value of 10) papers from each department, as selected by that department. The inclusion of a paper in these "top 10" lists would not necessarily mean that the paper has been peer-reviewed in any way. It would just signal that this paper may be worth checking out.
3. The purpose of journals/editors could shift from reviewing/publishing papers to aggregating all these informal announcements as well as all the "top 10" lists to identify most interesting/promising new papers in respective fields, with their URLs. Then it'd be up to the members of community interested in those topics to check out or not check out the papers in question.