User Password
|
Login / Create Account
<< Back to the Blog

In defence of open peer review

Tags open peer review
By: Rafael Najmanovich  
November 7 2012


I wrote the following text from the perspective of being a PLoS ONE academic editor, however, it clearly applies to any journal. Last week was open access week, to keep the moment going, here are my two cents. I am sure that I am not the first to suggest what follows but frankly I didn't have the time to go search. Apologies for any unassigned credit.

Whenever I review a paper, I stand by my comments and would have no problem if the authors knew my identity. Further still, I wouldn't mind if the whole community saw my review. While reviewers and editors strive to detect and prevent the publication of articles with errors, a lot of what is published, even in PLoS ONE, is still probably wrong. Many scientists and certainly a lot of people in society at large, assume that all peer-reviewed published papers need to be considered 'true' in some sense. As if the peer review process were some sort of final word.

One of the advantages of PLoS ONE over other journals is that the importance of a paper (not how much it is cited) is supposed to be judged by the community after publication. Somewhat in the lines of Arxivs.

I believe that if PLoS ONE were to make reviews and editor decision letters public, even if (although not ideally in my view) maintaining reviewer identities anonymous, it would add to the quality of PLoS ONE as it would make possible for the community to take in consideration (and criticize) reviewer and editor comments, adding to the openness of the peer review process.

This would have some desirable unintended consequences. First, reviewers and editors could start to be assigned a quality factor that may influence future assignments relative to the subject areas that they say to be experts in. Second, in the case of non-anonymous reviews, it will make possible to openly assign credit to reviewers for the important work that they perform - with the possibility also that the cumulative effort of a reviewer be taken in consideration in any judgements related to career advancement.

Secrecy is often cited as necessary to allow reviewers to be objective and fair in their review without fear of feuds or revenge in case of harsh but necessary comments or rejections. However, more often than not, secrecy allows reviewers to get away with unfair demands/attacks: 'i am given hell to publish, will do the same', 'i have an undisclosed conflict of interest and will review this paper anyway to advance my own interests', etc. While it is a responsibility of the editor to prevent these problems to happen, some undoubtedly pass unnoticed. The advantage of making reviews known is that it may allow readers to judge for themselves the demands of the reviewer and identify attacks. Further, if the reviewer identities are known, this may prevent all these problems with secret peer-review to happen in the first place.

In my view the problems with secret peer-review out-weight the benefits. However, it is so engrained in scientific publishing culture that a good first step would be to make reviews public even if reviewer identities remain secret (for the time being).

Any comments, suggestions, critics?



Comments (2)
Login to leave a comment.
matthieu chartier - November 12 2012, 10:31:29
I totally agree with reviews being public for the simple reason that it will give incentive to scientists to be more engaged in the progression of ideas of their respective fields. It will for sure create more discussions and interactions.

As for non-anonymous reviewers, I don't see why we shouldn't give it a try.

Science will inevitably be more open and transparent in the next decade, like many other things in society. Let's be the pioneers!
JEanne Garbarino - November 7 2012, 14:50:39
I agree with having an open peer-review system. I think the secrecy issue can actually lead to being less objective in some instances, allowing reviewers to be "meaner" than they would normally be if commenting using their identity. (By "meaner" I mean going overboard with unrealistic demands for revision, usually experimental in nature.) I am all for transparency and accountability, in every aspect of science.