Peer review in Peerage of Science consists of four stages. Each stage has a deadline, set by the Authors upon sending their manuscript and displayed to Peers deciding whether to engage as a Reviewer. Deadlines are automatically enforced once the process starts, it is not possible for anyone to ask for an extension. When a stage is completed before its deadline and the process continues to the next stage earlier than expected, the deadline of that next stage does not change.
Stage 1. Submission and Reviews
Authors submit a manuscript to Peerage of Science, where it will be available for other Peers to engage as Reviewers. Peers voluntarily engage in review processes to gain the benefits Peerage of Science provides for them (see For Reviewers).
Engaged Peers (hereafter Reviewers) have time to send their anonymous review until the time window for Stage 1 automatically closes. The review has to include a Peerage Essay, a max 1000-word text written in a standardized format, carefully enough to withstand peer review itself (see Stage 2). The review can optionally also include a more informal "additional comments" of unlimited length, for example containing line-by-line comments about writing, or more detailed suggestions for analysis methods, etc.
As soon as a Reviewer sends the review, Authors and the Editors tracking this process are notified and can access the review.
Stage 2. Peer-review-of-peer-review
From now on, the process can be accessed only by Authors, Reviewers, and Editors. Reviewers are notified that they can now evaluate the reviews sent by others. The evaluation is based on the Peerage Essay (see Stage 1). The evaluation gives 1-5 scores for reviewer's justification and accuracy in identifying manuscript strengths in Merits, weaknesses in Critique, and the scientific significance, implications and potential improvements in Discussion. Reviewers can also leave written feedback about the reviews.
Upon the deadline, or as soon as everyone has completed the evaluations, each review gets a quality index (Peerage Essay Quality, PEQ), according to the evaluations. Authors and tracking Editors are notified that the peer-review-of-peer-review is now complete and the scores and quality indices are displayed to them in the service. Reviewers see scores and indices only after Stage 4, so that their judgment remains independent.
Stage 3. Manuscript revision upload, or withdrawal for re-submission
By the deadline of Stage 3, Authors either send a revised version of the manuscript, or decide to withdraw the manuscript for later re-submission. When revising the manuscript, Authors now know the PEQ of each review, and are able to focus on reviews deemed most important in the peer-review-of-peer-review (see Stage 2). Authors should give credit to individual Reviewers using their unique id number, where appropriate. If Authors choose to withdraw the manuscript for later re-submission, this peer review process is complete, and the Reviewers receive their PEQ-score updates immediately.
Stage 4. Final evaluation of the revised manuscript
Final evaluation is seven scores on different aspects of the revised manuscript (breadth, impact, originality, data, methods, inference, literature coverage), and a free-text comment directed to Editors (but displayed to Authors and other Reviewers as well), where the scoring should be justified and any remaining issues stated. As soon as each Reviewer has completed the evaluation, or at the latest when the stage closes, the peer review process is completed. A quality index (Peerage Article Quality, PAQ) is calculated for the manuscript, according to the final evaluation of the revised manuscript (a Reviewer’s evaluation has weight proportional to the peer-reviewed quality of the review; see Stage 2).