Traditional peer review is still an important part of most branches of science. There are some branches where other mechanisms are advancing (especially in physics, where "crowd sourced" review is making big inroads). The idea behind peer review is simple enough: anonymous, qualified reviewers analyze papers before they are published. If a paper passes review (perhaps with changes or corrections), then it is published; otherwise, it isn't.
But the peer review process itself is obviously not perfect. It has lots of room for bias and subjective decision-making, and a strong bias toward orthodoxy. There have been some studies of the peer review process in the past, and they have shown some problems. Now there's a new study, quite clever, that shows quite convincingly the existence of a bias favoring prestigious institutions and individuals – a very human bias, one we can all understand, but simultaneously one we'd hope science would be free of.
This study started with a group of papers published in prestigious journals, written by the psych departments of prestigious institutions. These papers were modified to show them as coming from fictitious institutions and individuals. No part of the substance of the study was changed, not a word. Then these modified papers were submitted for publication to the same journal that had originally published them.
The first interesting result was that only 8% of the submissions were detected as duplicates. The screening process obviously isn't very robust.
The second interesting result is that 89% of the resubmitted papers were rejected. The primary reason given: “serious methodological flaws”. The simplest explanation for this result is that the prestige of the original submitters affected the reviewer's judgment – exactly that bias that we'd wish wasn't there.
Scientists are humans, too. And peer review isn't all that great a process...
No comments:
Post a Comment