By Michelle Spektor
Peer review has long been the standard for quality science, but recent concerns about fraud among authors, bias among reviewers, and possible hindrance of scientific progress has led some to question the effectiveness of a process that relies on anonymous experts and occurs behind closed doors.
Panelists who spoke Feb. 18 at the American Association for the Advancement of Science meeting in Vancouver agreed that peer review needs improvement, but offered different perspectives on whether a more open system is the answer.
“There is more of a demand for transparency in the peer review process,” said Linda Miller, a professor at the New York University Langone Medical Center and a former editor at Nature Publishing Group. Miller noted the mismatch between the confidentiality of peer review and emerging societal values of information-sharing.
Some journals have already adopted more transparent peer review processes. Atmospheric Chemistry and Physics engages in “open peer review” by disclosing the identities of reviewers. The two-year-old EMBO Journal is even more transparent, as its editors publish written comments and correspondences from all rounds of the review process along with the articles themselves.
Miller believes that a system like EMBO Journal’s would encourage reviewers to produce better reviews and put pressure on editors to hold reviewers to higher standards.
Emilie Marcus, the CEO of Cell Press and editor of Cell, was more skeptical of the benefits of a system that holds reviewers accountable to the public in addition to their editors.
“I am a little less convinced of the value that comes from unearthing all the comments and leaving it up to individuals—whether they are journalists or readers—to then sort out, essentially, did the editor make the right decision in deciding to publish this?” she said.
Additionally, Marcus noted that reviewers may not feel comfortable with the publication of their reviews. An open peer review process like EMBO Journal’s would also require publications to develop a process for editing and releasing reviewer comments—a task that Miller said many journals might find difficult to take on.
Meanwhile, journals and scientific disciplines that engage in traditional peer review have developed their own ways to optimize the process. The 29 publications of Cell Press are among the few that use a two-tier review process; papers are first internally reviewed by staff editors and then externally peer reviewed by scientific experts. In addition, reviewers can look at and evaluate other reviewers’ comments. That method that helps reduce reviewer bias, Marcus said.
Articles submitted to journals in the astronomical and physical sciences disciplines are usually pre-reviewed by other members of the field through a pre-published paper database called ArXiv.
“There is a culture of preprints in a number of the physical sciences, so a lot of research reaches the community before it’s published in the formal journals,” said Chris Biemesderfer, the director of publishing for the American Astronomical Society. “That’s one of the ways we know we’re not concerned about peer review slowing down the access to new results.”
While peer review faces challenges, scientists who partake in the process generally approve of it. A 2007 Sense About Science survey of peer review showed that 91 percent of scientists think peer review improves their own work.
Panelists also agreed that peer review is still the most preferred method of ensuring that only the most impactful, innovative and scientifically legitimate research reaches publication—at least for now.
As Miller asked: “There are dissatisfactions with peer review, so what would you change in regard to trying to improve a system that we all acknowledge isn’t perfect but is the best one so far that we’ve come up with?”
Michelle Spektor will graduate from Cornell University in May with a bachelor’s degree in biology and society. She is also an intern at the Cornell Chronicle and previously wrote for Science Progress. Reach her at mjs477@cornell.edu.
