A free service for scientific peer review and publishing

much ado about noting

Ego eimi?

Janne-Tuomas Seppänen
10 Jun 2012

Biology Direct is a journal using open peer review (open as in "public and signed", not as in "free-for-all") to "eliminate the principal sources of abuse in the refereeing process". To be accepted for publication, authors must succeed in soliciting signed peer reviews from at least three editorial board members. The reviews are published along with the article. The rationale is that responsibility for the review is increased, and unimportant research does not manage to solicit three signed reviews. The journal is doing quite well, so it seems to work. Should anonymity in peer review be history?

Peerage of Science is not employing open peer review (open as in "public and signed", but yes as in "anyone qualified can participate"). Though the system is designed to encourage reviewers to disclose identity after the process, during the process everyone is expected to be anonymous. Even editors tracking a process are anonymous unless they make a publishing offer, and blind to the identity of both the authors and the reviewers. Would science not be better with total openness? Have we let the fount of evil run rampant in Peerage of Science?

Well, I used to sign each and every one of my reviews as a matter of personal policy. I was also eager to announce that policy to anyone who would bear to listen to my noble motivations for not wanting to be anonymous like the old people. I felt I was doing the right thing. I enjoyed the breeze, standing in solitude on the moral high ground (looking back, and around, I notice the highlands are actually rather crowded with fellow young PhD's). I also not-so-implicitly argued that signing my reviews not only forces me to try to be justified and thorough, but that signing actually was proof of that.

Private motivations were somewhat less noble. I provided my name hoping... to get a name. To gain existence. I desperately wanted the authors and other reviewers to know those arguments were mine. Along with my own research, peer reviews were one way to get from being nobody to being somebody. If my arguments were solid and insightful, and more people read them with my name attached, my name would start to carry more weight. I wanted prestige.

Hail the real beast. Prestige is the principal source of abuse in the refereeing process.

Someone with more of it, or with a position in an institution with lots of it, can more easily get away with using prestige instead of arguments to justify judgment as a reviewer, or justify their disregard to judgement as an author. Not to mention the different chances of getting your submission into peer review, or having the editor's ear for underhanded comments not disclosed to authors. Or at least, that is what many of us without prestige suspect might occasionally happen, especially in the most prestigious journals. That suspicion alone is bad for science and scientists, even if misconduct never really happens.

Peer review should be, by definition, judgement by equals. That is why reviewer and author anonymity is emphasized in Peerage of Science. That is also why reviewers in Peerage of Science have been quick to criticize the few authors who (usually accidentally) left their names on the first page of the manuscript. Reviewers are questioning whether author's disclosure of identity is openness, or an attempt to add weight to the research by using the argument "because I am..."?

On the other hand, Peerage of Science stands or falls by the trust editors can place on the peer reviews. The service does its best to increase the trustworthiness of the anonymous reviews:
1) the right to review is granted only after identity, and proof of at least one published article in a peer reviewed journal, have been validated, and
2) editors can view the anonymized profiles of the reviewers, and profiles will be a reliable indicator of both quantity and quality of reviewing experience - in a few years.

But in these early days most of the anonymized profiles offer little information besides self-declared expertise of the reviewer. Some Peers have just published their first research, while some have been leading research groups and judging research for decades. It is understandable that editors evaluating the usefulness of the new way would like to know who writes the reviews.

Hence, a compromise: editors now have an "Ask identity" -tool next to each peer review. Clicking it sends an email to the reviewer, telling who is asking and providing a link to click, if reviewer agrees to disclose identity. Editors can also choose to embed this link in all the referrals they make (i.e. when they use the tool to request reviews from trusted reviewers). If reviewer agrees, that particular editor then sees the reviewer's name in the process view, but for that process only. Reviewer continues to be anonymous to everyone else, and in all other processes. Reviewer is of course free to ignore the request as well.

This naturally summons some of the demons of the old system. You can't be certain that editor's decision is not influenced by reviewer prestige, and a private correspondence channel is opened between the reviewer and the editor. However, looking at the 80+ peer reviews received so far and the PEQ-scores assigned to them, gives confidence that quality control will be maintained by the peer-review-of-peer-review. Other reviewers will continue to evaluate you, and authors will address or ignore your comments, without knowing how important you are.

The choice is yours, as a reviewer, and as an editor. Click as you see fit.

About Janne-Tuomas Seppänen

I wonder if the theme song of Casino Royale reflects the issues discussed here?


« | »

peerage of science Facebook peerage of science twitter peerage of science twitter