An article in Inside Higher Ed, Puzzling peer reviews, highlights supposed dangers of open access publications. John Bohannon of Harvard University wrote a deliberately flawed biology article using a fictitious name and non-existent institution and submitted it to over 300 open access publications. It was accepted by about half of them (read a longer description of the experiment can be found in Science, Who's afraid of peer review?). The article was written as part of a survey to see how much peer review was involved in the rapidly expanding open access journal market and the results cast a serious shadow over many of them. It contained serious scientific flaws that would be obvious to any academic in the field so the journals who accepted it had clearly not carried out any sort of serious peer review. Interestingly it was not only obscure journals that failed the test, even journals run by the big academic publishers fell for the trap.
"Acceptance was the norm, not the exception. The paper was accepted by journals hosted by industry titans Sage and Elsevier. The paper was accepted by journals published by prestigious academic institutions such as Kobe University in Japan. It was accepted by scholarly society journals. It was even accepted by journals for which the paper's topic was utterly inappropriate, such as the Journal of Experimental & Clinical Assisted Reproduction."
This is clearly unacceptable, though I find it hard to believe that so many people would fail to at least check the credentials of Ocorrafoo Cobange of the Wassee Institute of Medicine in Asmara. The basic principles of Howard Rheingold's Crap detection seem to have been overlooked here (Is this a credible source? What other publications come from this person/institution?). But does this mean that the open model is inherently flawed and that the traditional closed journal system is more credible?
All journals, open or closed, rely on peer review for the credibility of their publication. In neither model do the reviewers get paid for their services so the crucial factors must be the integrity and experience of the reviewers and the rigour of the process. Those were clearly lacking in the journals that accepted this fake article but I wonder what the success rate would have been if submitted to a large number of traditional journals. I suspect that even there there are some whose quality assurance processes are inadequate.
This is not really an openness issue but more a quality issue that can affect any publication open or closed. Those whose peer review and editing routines are not sufficiently rigorous will lose credibility. The reason so many open access journals seemed to fail this particular test could be because there are so many new journals on the scene who are still trying to establish themselves and have not developed quality assurance routines in their peer review processes. There are of course many extremely dubious publications out there, often based in developing countries but with titles that begin with "The American Journal of ...". These offer publication at a price and often succeed in luring academics into paying for publication. These deserve to be named and shamed as they are in Jeffrey Beall's List of predatory publishers 2013.
However it is important that we are not be tempted into thinking that openness means poor quality. The two are simply not related as David Wiley points out in an article, On quality and OER. Copyright or a price tag are no guarantees of good quality, whether we're talking about educational resources or academic publications. There are excellent open resources and excellent proprietary resources and their excellence depends on quality assurance routines and professionalism.
"Because quality is not necessarily a function of copyright status, neither traditionally copyrighted educational materials nor openly licensed educational materials can exclusively claim to be “high quality.” There are terrific commercial textbooks and there are terrific OER. There are also terrible commercial textbooks and terrible OER. Local experts must vet the quality of whatever resources they choose to adopt, and cannot abdicate this responsibility to publishing houses or anyone else."
No comments:
Post a Comment