Peer reviews in Peerage of Science have two required items. One is a categorical revision recommendation to the authors, and the other is an essay. The first uses soothingly familiar terms from the publishing empire, like recommending (but note, not requiring) a major revision. But the concept of an "essay" in the context of scientific peer review often generates questions, expressions of uncertainty, and even suggestions to abolish it entirely and follow more orthodox concepts. What is this essay thing, why should it have a place in scientific peer review?
Michel de Montaigne called his writings "Essais", i.e. attempts, and so did Francis Bacon. Immanuel Kant used an essay to declare that enlightenment is the freedom - and courage - to think for yourself. No, essays in Peerage of Science are not meant to be philosophical treatises with influence lasting for centuries. But etymology, and anecdotal bridges to what later generations think some long-dead philosopher meant, do lead, as always, to a useful distillation of what things ought to be.
Essays in Peerage of Science are, as Michel said, attempts. Attempts to convince the reader that a manuscript should be casted as science by publication in a peer-reviewed journal. Or attempts to convince reader that the research in question has no hope of attaining the coveted halo of science no matter how much the text be revised. Or, hopefully, attempts to briefly and lucidly explain both the merits and the shortcomings of a manuscript, and discuss the importance, implications and suggestions. Attempts to show the reader the expertise of the writer, showing your ability to think, in public.
Peer reviews in Peerage of Science are attempts, because they are judged. Because they may be deemed unworthy of further attention. Because they may lead to well deserved recognition.
And that is the essence of essays for scientific peer review. The form is nothing but a servant of the goal. Essays need to be of limited length, so that Peers can read and judge several of them, and so that Editors can effectively utilize them to gain an understanding of the value of a manuscript. Essays need to be written in full sentences and paragraphs, with some structuring readers recognise, so that the writing can be understood, then judged, then utilized, without much deciphering. It is good if the quality of writing makes reading peer reviews at least painless, if not enjoyable.
So, Peerage of Science now has a new review submission interface, designed to encourage the form and thus the goal of essays in peer review. There is a simple choice menu for the revision recommendation, and then a text editor where the essay plus additional optional text can be written or pasted. The text editor features basic formatting tools and ability to include figures, and users create and preview a pdf directly from the editor. No more submitting pdf's.
The text editor contains a template for organising the essay, with the following four sub-headings:
- Introduction, to briefly describe the main result or content of the manuscript.
- Merits, to describe what is good in the manuscript, or to state that there are no merits.
- Critique, to describe the weaknesses of the manuscript, or to state that there are none.
- Discussion, to elaborate e.g. on the importance and implications of, or improvements for, the manuscript.
These four sections together must total 1000 words or less. Reviewer can also add for example line-numbered comments, informal suggestions, or more detailed analysis recommendations under the heading Additional comments for authors, and the length of this section is not limited. Additional comments are displayed on the same review pdf, after the essay. Reviewer also has the option to add a supplementary file, for example an annotated copy of the manuscript, or an article authors should read.
It is true that rigidly standardized organisation of the peer review text has its downsides. Experienced reviewers might have developed an established structure of their own, and the Introduction-Merits-Critique-Discussion organisations then feels disruptive. Others might feel uncomfortable compartmentalising their writing in this way, preferring a more free flow of thinking and writing at the same time.
However, even those who would prefer not to follow the organisation, will surely find it very useful and time-saving when evaluating the reviews written by others. The three evaluation categories - Merits, Critique, Discussion - correspond to well-defined sections of the text and are always found at the same place. If you are an editor, you will find the new essay organisation vastly superior to traditional reviews. It is much easier to use these structured essays to quickly assess what the reviewers think the main strengths or weaknesses are. Authors can use the essays to easily build a list of merits they perhaps should emphasise, and a list of challenges they need to address in a revision.
Think of this organisation analogous to the standard scientific article format (without a methods-section): first introduction, then brief, factual description of your findings, and finally a discussion on what all that means.
I dare say the fact that reviewers know Peers will read and judge their text has had a very favourable impact on peer review in Peerage of Science. Most of the 56 reviews received so far are thorough, stringent (getting praise from Peers is by no means easy), yet fair evaluations of science, and those few that are not, have been duly assigned low PEQ by other reviewers. Further enhancing this by having standardized structure to essays will make Peerage of Science peer review even better than what it already is.
The strong anonymity, and freedom to choose what data you show in your profile, ensure that in Peerage of Science judgement never entails public embarrassment. But essays here nonetheless are a form of thinking in public. As Kant and Horace said: Sapere aude!
One of the founders of the service, and a postdoctoral researcher at University of Jyväskylä, Finland.
Every attempt I've made to understand Kant ends up in a transcendental dialectic illusion, so I have given up.
But I do want something he had. He immediately stopped publishing and attending scholarly gatherings when he got his professorship, for 11 years. No, he was not fired, nor pressured to resign by the university. That academic freedom and security I want.
Had Kant lived in a world with modern academic policies, we would have 20+ long-forgotten least publishable units, instead of the Critique of Pure Reason.