The WEAK case against double-blind peer review – highlighting why we need it!! #science @NatureNews

NATURE this week feature a correspondence from Thomas E. DeCoursey reasoning against double-blind peer review. In my humble opinion his reasoning is flawed…. not unlike the current peer-review structure. To air out my laundry, I support a completely open or double-blind system for manuscript peer review. All of the peer review models have some flaws, but these two seem infinitely better than the current system where authors are blinded to reviewers but not vice-versa.

DeCoursey makes the somewhat legitimate point that it may be possible for reviewers to ascertain who the authors of a manuscript are based on citations. However, there would always be some element of doubt for the reviewer about who the authors are, and there are many cases where this circumstance would not occur.

Then DeCoursey reasons that reviewers need to know who the authors are in order to judge them on their past work…. or something…. wha???

To function in our increasingly competitive research culture, in which misconduct is on the rise, researchers need to be aware of which labs can be trusted and which have a record of irreproducibility. If a highly regarded lab and one with a questionable reputation each submit reports of similar investigations, a good reviewer would be extra vigilant in assessing the less-reliable lab’s study, even though the same evaluation standards would be upheld for both.

Yes, misconduct is on the rise, but this point seems wrong to me on every level. Reviewer’s should be vigilant of misconduct and scientific quality on every paper, regardless of what lab the paper comes from. Plenty of ‘good’ labs have had to retract papers for many reasons, and labs with a history of misconduct have reformed and redeemed themselves with quality papers. In fact, less vigilant reviewers may be to blame when flawed papers from highly regarded labs make it through the review process with glaring mistakes. Any reviewer that is more or less vigilant reviewing a manuscript based on the author’s names is not an impartial reviewer. PARTIALITY is bad when reviewing papers and grants…  Ethics 101 – Conflict of Interest. For the same reason, most journals won’t allow scientists to review a manuscript from within the same institution.

There is a reason double-blind experimental design is the gold standard for experiments and human clinical trials. Just like a reviewer might think he knows who the authors are, a doctor might think he knows whether a patient is receiving placebo, but neither can ever really be sure. Why wouldn’t we want the same type of controls for peer review?

Double-blind peer review removes this crucial quality-control option, opening the way for mediocre and bad labs to clutter the literature with sub-standard science.

#FacePalm…

Maybe I’m jaded, but good reviewers should be screening out sub-standard science regardless of whether they know what lab a manuscript is from or not. This closing statement makes it sound like DeCoursey thinks only the best labs, with the biggest names, and the highest impact factor publications should be publishing… which I hope is not the case (maybe I read into it too much). If it is the case, then that only argues stronger for a double-blind peer review system.

And in closing, a double-blind peer review system would help avoid racist, sexist, or other embarrassing situations like this one, where a reviewer commented that the two female author’s should add a male author in order to strengthen the manuscript. Double-blind peer review erases sexism, racism, nationalism, institutionalism (?), and other discrimination from the peer review process, which is definitely huge plus!

Anti-Science Quotable: Michigan Representative Dan ‘Im-a-scientist’ Benishek thinks climate science is bunk

banishek

Michigan GOP Congressman Dan Benishek, a former general surgeon, claims he is a scientist and doesn’t believe the many peer-reviewed scientific studies on climate change… even though he believes in peer-reviewed science. He also says, ‘well’ a whole heck of a lot. Check out the video on Huffington Post here.

The climate may be changing, but I don’t think man is contributing to it. Well, I think it’s just the natural course of things. … there’s no scientific evidence that shows any of that. I’m not sure there’s any any evidence to prove that there’s man-made catastrophic global warming. Well, there’s no significant scientific evidence. Well, I am a scientist. You know, I believe in peer-reviewed science. But, I don’t see any peer-reviewed science that proves that there is man-made catastrophic climate change.

Scientists, what happens when we stop trusting each other? #science

neuroskep

Check out this great post by Neuroskeptic, which addresses the issue of trust between scientists. While scientists are supposed to be skeptical of conclusions implied from published results, we need to be able to trust the methods and results themselves. 

Science fraud, questionable research practices, and replication have got a lot of attention lately. One issue common to all of these discussions is trust. Scientists are asking: can we trust other scientists to be honest? Is peer review based on trust? Is the act of discussing these issues itself eroding trust? What can we do to restore trust?

Times are a changin’: Testing out double-blind peer review in scientific publishing #abouttime

cressey

 

Daniel Cressey has written a great article in Nature about a double-blind peer review system for scientific publications. In the current system of peer review (see this post for more info on peer review), most reviewers know the names of the authors, but not vice-versa, which many people believe means reviews are not un-biased. These biases can be conscious or subconscious, against known authors, minorities, women, and others (ie. younger scientists).

But last week an article in Conservation Biology1 revealed that journal would be considering ‘double blind’ peer review — in which neither the reviewer nor the reviewed knows the other’s identity. Double-blind peer review is common in the humanities and social sciences, but very few scientific journals have adopted it.

In addition to Conservation Biology, a few other publishing journals that are trying out double-blind peer review, including two from Nature Publishing Group, Nature Geoscience and Nature Climate Change. While it is too early to know how this system will work, it is definitely exciting to see some change to this system. Definitely check out Cressey’s article! And check out CauseScience’s many posts about peer-review and what is wrong with it.

Retraction of science papers: Is peer-review to blame??

 retraction

Nikolaus Kriegeskorte has written a very thoughtful and intriguing article about the recent explosion of scientific publications retractions (Check it out at The Conversation). The article discusses the possibility that the current, but outdated, peer-review system may be to blame for the trend in retractions. The article highlights that despite the bad press for science, retractions highlight the positive and negative side of science.

Retractions reveal both science’s weakness and its strength. Science frequently goes wrong; that’s its weakness. Then science corrects itself; that’s its strength. And yet there’s a lesson in the rising rate of retractions.

Kriegeskorte points out that the current peer-review system allows ‘the world’ or ‘the public’ to see science (in the form of press releases etc) before the scientific community as a whole has a chance to judge the science. He suggests a better system.

Back when articles needed to be printed on physical paper, we needed to filter before publication to control the costs. Today the internet enables us to “publish then filter”, to use Clay Shirky’s useful phrase. This will revolutionise scientific publishing.

While implementation of a system like this with the current scientific model may be difficult, I can’t help but agree that the peer-review system is broken and needs updating. If for no other reason than to save the image of scientists and science as a whole. But I don’t doubt it would also result in better science.

What does it mean when scientists say peer-reviewed?? #academiclife

Want a good explanation of what peer review is? How it works? What the process is like? The Conversation has posted a terrific description of peer review basics. The authors (Andre Spicer and Thomas Roulet) also give a nice summary of the benefits of peer-review, but also the problems with the current peer-review system. Check it out!

Peer review is one of the gold standards of science. It’s a process where scientists (“peers”) evaluate the quality of other scientists’ work. By doing this, they aim to ensure the work is rigorous, coherent, uses past research and adds to what we already knew.

NIH contest to improve peer review: win $10,000

 NIHreview

The NIH center for scientific review is hosting a contest to get ideas for improving the peer review system to identify potential bias. For each of the 2 challenges, the first prize is $10,000 and second prize is $5,000!

Challenge #1
New Methods to Detect Bias in Peer Review
Submit your idea on how to detect bias among reviewers due to gender, race/ethnicity, institutional affiliation, area of science,  and/or amount of research experience of applicants. First and Second prizes will be offered in two categories, best empirically based idea and most creative idea. Additional details can be found at FRN Doc.2014-10196

Challenge #2
Strategies to Strengthen Fairness and Impartiality in Peer Review 
Submit your idea on how to strengthen reviewer training methods to enhance fairness and impartiality in peer review. First and Second prizes will be offered for the best overall ideas. Additional details can be found at FRN Doc.2014-10203.