Two weeks ago I was in Ottawa in a conference, and among other interesting things (never guessed I'd march to protest against a corrupt government with 2000 other scientists), I attended a talk by Rosie Redfield about the arsenic life debacle. If you are the kind of person reading this, odds are you are also the kind of person who already knows the story, but in short: NASA researchers announced with Much Adoe the discovery of a life-form fundamentally different from all other life on this planet, but then in post-publication peer-review many scientists convincingly showed it was (probably) About Nothing. And there was certainly also much ado about noting, and about being noted, mostly in twitter.
The story can be seen as a triumph for the scientific community: some of us did research, published it, and then some of us looked at the data, analysis and argument, found those to be insufficient to support the inference presented, published the counter-arguments, and so truth was served, progress of science protected. This is surely how we scientists like the story to be presented. See, you laymen; this is science, this is what we are, this is what we do - we, and, by extension, science itself are self-correcting, self-governing, concerned with facts not personas, vigilant for truth, and right. Right. Right. Right?
As a cynical member of that community my view of us is not quite so rosie. Listening to the talk (and that other one about scientific careers available in Hollywood re-creating physiology of dragons and griffons) got me thinking about some more mundane aspects of our profession, and about its eponymous vessel of distribution (the nature of other such containers is certainly implicated also).
First, I revel the day when scientists proving their point actually get to do stage-diving and crowd-surfing in scientific conferences. It was clear I am not the only one who would wish to. Personally, I perhaps would try to avoid singing "you Fail! you Fail! And you too, Fail! Me, Epic Win!" while surfing, but probably could not resist the temptation either.
Secondly, it is painfully obvious how heavy and sharp the sword of Damocles that hangs above a scientist really is. But note, usually that sword which is supposed to hang by a hair is tightly affixed by ink, or, nowadays, by a doi.
In this case the blade dropped only with a thorough post-publication peer review - duly received thanks to the claims being so extraordinary, and promoted aggressively with the PR-budget of NASA, and being published on a notable platform. Pre-publication peer review, of a journal named after the pursuit of knowledge itself, failed to behead the king's courtier before the coronation. How many scientific results published today in peer-reviewed journals would melt equally fast if subjected to thorough scrutiny by so many peers? How many of those faulty papers will never be challenged? How many of those will be used, as references counted but never read, to justify research grants and appointments for years to come, displacing others?
Imagine that papers announcing the discovery of new life-forms, pink unicorns, trustworthy prime ministers, and other such wonders, would always be first submitted to pre-publication peer review in a service like Peerage of Science. A service where any validated scientist can freely engage as reviewer and anonymously challenge the result while knowing that their arguments, too, will be subject to judgement by peers.
Freely engageable, cross-reviewed, measured and rewarded pre-publication peer-review process created by all of us together, rather than by a single editor and two un-accountable reviewers, would be much, much better for science. And of course recycling is better for nature too.
One of the founders of the service, and a postdoctoral researcher at University of Jyväskylä, Finland.
Shakespeare was known to feature bloggers in his plays.
Balthasar, the king's courtier:
Note this before my notes:
There's not a note of mine that's worth the noting.
- Much Ado About Nothing, Act II, Sc.3