Last week, I had the opportunity to participate in the "The Future of Peer Review" -seminar organized by ALPSP. With a (paying!) audience of almost 80 publishing professionals, the seminar was drawing quite a lot of attention. There were very interesting talks, three of which I want to highlight in particular. I actually ended up having to change my view on some things, which is a sign of a very successful seminar indeed.
The thing in question was the relationship between journal rejection rate and scientific quality - Ulrich Pöschl made a compelling argument that the correlation is actually negative: journals that reject large percentage of submissions do so because they have to, due to the low quality of submissions they receive. High rejection rate simply means poor manuscripts get sent to the journal in hope that they make it through, suggesting authors of low-quality science perceive the journal willing to publish lower-quality material.
I was also very impressed by what Ulrich's journal Atmospheric Chemistry and Physics has achieved in a short time, and moreover impressed by how innovative and effective their policies leading to that success have been. There is certainly lot to learn for any journal editor or publisher from them. Ulrich's talk is available as audio and pdf of slides are also available
Another particularly good talk was by Cameron Neylon. He noted that despite the amount of chatter about peer review, and its importance to science, there is remarkably little (if any) science about peer review! What we have is plenty of different surveys documenting what we believe about peer review, but no quantitative science whether and how peer review works to deliver what it is supposed to deliver. The surveys about peer review to date are quite like asking people who use homeopathic "medicine" what they feel about effectivenes about said concoctions. Do you think mercury dilution is effective? How do you feel we should improve mercury dilution mechanisms?
The third talk I want to highlight is, quite naturally, my talk on Peerage of Science (hey, it's my blog post, you didn't read this far expecting to get away without shameless self-promotion, did you?). In particular, I would like all scientists to pay attention to the "three exclusive things" -part in the beginning of the talk. The next time a journal asks you to do a peer review for them, pause for a moment to think what that request actually entails. Pause for a moment to think through if you want to continue supporting that, now that you have an alternative also.
The talk is available as video, editing together the animated slides and the audio recording. My first-ever youtubing, so I'm catching up with the internet of 2005! The video includes questions from the audience.