Academic Research's Incentives Have Drifted from the Pursuit of Truth
Incentives in Academic Research
Drawing on Charlie Munger's insight about incentives, a SAT researcher argues that academic research has strayed from its core goals of advancing knowledge and training honest scientists. He shares personal encounters with researchers who, after being shown that their results are wrong or misleading, refuse to retract or correct them—prioritizing publications and career advancement over scientific integrity. The piece calls for a renewed commitment to correctness and curiosity.
The issue is, once you published the paper, and got the promotions and fame, it doesn’t matter that the results are wrong, and known to be wrong or misleading, and potentially doing harm to the advancement of science. It’s not your problem. It’s someone else’s problem.
- t0mpr1c3
I would add that the problem is not new, and not limited to junior researchers.
In my first faculty position about 20 years ago, I followed a line of research that a senior (MacArthur Fellowship-winning) colleague had pioneered. After three years and a few negative results, I came to suspect that several high-profile publications were p-hacked and likely false positives.
It was extremely demotivating, to say the least. I switched to a different field and fairly soon afterward left academia altogether.
A very senior scientist later confirmed my suspicions about their disregard for professional ethics.
- senshan
Hmm... The author starts with C.Munger's quote “Show me the incentive and I will show you the outcome”, then proceeds to express dissatisfaction with the current outcome of academic research. Regrettably, the author never returns back with:
a) analysis of current incentives; and
b) proposal to change those for a better outcome;
- smath
There is a great blog post [0] that sohl dickstein had a couple years ago, about what happens when the metric being optimized is only a proxy for the thing you want to maximize — what happens eventually is “overfitting”. The original metric (which we cannot directly optimize for) actually starts to get worse.
I believe publishing papers and citations are those proxy metrics.
[0] https://sohl-dickstein.github.io/2022/11/06/strong-Goodhart....
- bonoboTP
I think everyone who enters academia goes through this phase of disillusionment. This thing you revered so much as this temple of knowledge turns out to involve the same kind of human business you see anywhere else, with petty fights, little cliques opposing each other, grudges, backscratching, quid pro quo etc etc.
I think once you come to terms with this, you come out on the other side realizing that this is nothing new. The idealized version never existed. What's new is the scale and all the quantitative metrics regarding publication.
> Academic research was meant to be about breaking new ground, keeping to honesty and good scientific conduct, being clear and upfront about uncertainties, errors, and mistakes, and improving our common understanding of science, all the while training the new generation to follow these goals and principles.
This is a naive idealism. Research takes effort and someone has to pay. Sometimes this someone's may profess beliefs that they only want to sponsor unrestricted objective inquiry, but everyone has biases and in an iterated game, soft preferences will be gamed too. In other words, there's is always an effect of "the one who pays orders the song".
The one who pays wants results. He wants prestige through the sponsored project. He doesn't want the researcher to just sit in a room and fail to generate results for years. He wants the researcher to have groundbreaking results. What they are matters less than that they be impressive and improve p […]
- SubiculumCode
The author provides no empirical evidence to support their claims, providing only anecdotal evidence. Their claim might be true, might be truer in some places/departments/countries than in others, but I do happen to agree with the author...or I am concerned about the issue.
Also anecdotal, as a peer reviewer, I too often encounter shoddy, careless, or even probably fraudulent conduct papers, and it frustrates me because peer review is not cost free: Authors are asking scientists to spend their precious time to understand the author's work, and that only works under a social contract....that authors only send out manuscripts for peer review after they have tried their best to do their best science.
I hate to say it, because I know that China has great scientists, but most of the manuscripts I receive and reject due to sloppy work, or purposeful negligence in citing most directly relevant to their manuscript (often because that literature did exactly what they are trying to pass if as new), or out-right making shit up, have been manuscripts from Chinese Universities. Yes it happens from western authors as well, but what I am naming is real, and is increasingly a major burden on peer-review as a scientific institution. So much so that many have confided to me that they will refuse to serve as a reviewer of a manuscript out of China unless it is from a known lab of some standing. In science, there must be a culture motivated not by number of papers published or grants funded, but […]