Archives des mots-clés : Open peer review

Retrospective analysis of the quality of reports by author-suggested and non-author-suggested reviewers in journals operating on open or single-blind peer review models

Statut

Objectives

To assess whether reports from reviewers recommended by authors show a bias in quality and recommendation for editorial decision, compared with reviewers suggested by other parties, and whether reviewer reports for journals operating on open or single-blind peer review models differ with regard to report quality and reviewer recommendations.

Design

Retrospective analysis of the quality of reviewer reports using an established Review Quality Instrument, and analysis of reviewer recommendations and author satisfaction surveys.

Setting

BioMed Central biology and medical journals. BMC Infectious Diseases and BMC Microbiology are similar in size, rejection rates, impact factors and editorial processes, but the former uses open peer review while the latter uses single-blind peer review. The Journal of Inflammation has operated under both peer review models.

Sample

Two hundred reviewer reports submitted to BMC Infectious Diseases, 200 reviewer reports submitted to BMC Microbiology and 400 reviewer reports submitted to the Journal of Inflammation.

Results

For each journal, author-suggested reviewers provided reports of comparable quality to non-author-suggested reviewers, but were significantly more likely to recommend acceptance, irrespective of the peer review model (p<0.0001 for BMC Infectious Diseases, BMC Microbiology and the Journal of Inflammation). For BMC Infectious Diseases, the overall quality of reviewer reports measured by the Review Quality Instrument was 5% higher than for BMC Microbiology (p=0.042). For the Journal of Inflammation, the quality of reports was the same irrespective of the peer review model used.

Conclusions

Reviewers suggested by authors provide reports of comparable quality to non-author-suggested reviewers, but are significantly more likely to recommend acceptance. Open peer review reports for BMC Infectious Diseases were of higher quality than single-blind reports for BMC Microbiology. There was no difference in quality of peer review in the Journal of Inflammation under open peer review compared with single blind.

URL : Retrospective analysis of the quality of reports by author-suggested and non-author-suggested reviewers in journals operating on open or single-blind peer review models

DOI : 10.1136/bmjopen-2015-008707

Diamond open access and open peer review: An analysis of the role of copyright and librarians in the support of a shift towards open access in the legal domain

Posté le par

“The aim of this paper is to support initiatives that stimulate volunteer involvement in creating qualitatively good conversations about the law on the internet. The article’s core argument is that policies on open access, copyright and library services all concentrate nowon the results of scholarly conversations, while a shift in focus towards the process of scholarly communication is needed to develop new incentives for a culture of sharing. Ways to foster openness in scholarly communication need to be discipline specific. This will be elaborated by discussing the plan for an open environment for collaboration on an English translation of a Dutch introduction to private law.”

URL : http://webjcli.org/article/view/302/421

Defining and characterizing open peer review a review…

Statut

Defining and characterizing open peer review: a review of the literature :

“Changes in scholarly publishing have resulted in a move toward openness. To this end, new, open models of peer review are emerging. While the scholarly literature has examined and discussed open peer review, no established definition of it exists, nor are there uniform implementations of open peer review processes. This article examines the literature discussing open peer review, identifies common open peer review definitions, and describes eight common characteristics of open peer review: signed review, disclosed review, editor-mediated review, transparent review, crowdsourced review, prepublication review, synchronous review, and post-publication review. This article further discusses benefits and challenges to the scholarly publishing community posed by open peer review, and concludes that open peer review can and should exist within the current scholarly publishing paradigm.”

URL : http://pdxscholar.library.pdx.edu/ulib_fac/1/

The four pillars of scholarly publishing The future…

Statut

The four pillars of scholarly publishing: The future and a foundation :

“With the rise of electronic publishing and the inherent paradigm shifts for so many other scientific endeavours, it is time to consider a change in the practices of scholarly publication in Ecology and Evolutionary Biology. To facilitate the speed and quality of science, the future of scholarly communication will rest on four pillars – an ecosystem of scholarly products, immediate and open access, open peer review, and full recognition for participating in the process. These four pillars enable us to build better tools to facilitate the discovery of new relevant work for individual scientists, one of the greatest challenges of our time as we cope with the current deluge of scientific information. By incorporating these principles into future publication platforms, we argue that science and society will be better served than by remaining locked into a publication formula that arose in the 1600s. It has served its purpose admirably and well, but it is time to move forward. With the rise of the Internet, scholarly publishing has embraced electronic distribution. But the tools afforded by the Internet and other advancing technologies have profound implications for scholarly communication beyond just distribution. We argue that, to best serve science, the process of scholarly communication must embrace these advances and evolve. Here we consider the current state of the process in ecology and evolutionary biology and propose directions for change. We identify four pillars for the future of scientific communication: (1) an ecosystem of scholarly products; (2) immediate and open access; (3) open peer review; and (4) full recognition for participating in the process. These four pillars will guide the development of better tools and practices for discovering and sharing scientific knowledge in a modern networked world. Things were far different when the existing system arose in the 1600s, and though it has served its purpose admirably and well, it is time to move forward.”

URL : https://peerj.com/preprints/11/

Open evaluation a vision for entirely transparent post…

Statut

Open evaluation: a vision for entirely transparent post-publication peer review and rating for science :

“The two major functions of a scientific publishing system are to provide access to and evaluation of scientific papers. While open access (OA) is becoming a reality, open evaluation (OE), the other side of the coin, has received less attention. Evaluation steers the attention of the scientific community and thus the very course of science. It also influences the use of scientific findings in public policy. The current system of scientific publishing provides only journal prestige as an indication of the quality of new papers and relies on a non-transparent and noisy pre-publication peer-review process, which delays publication by many months on average. Here I propose an OE system, in which papers are evaluated post-publication in an ongoing fashion by means of open peer review and rating. Through signed ratings and reviews, scientists steer the attention of their field and build their reputation. Reviewers are motivated to be objective, because low-quality or self-serving signed evaluations will negatively impact their reputation. A core feature of this proposal is a division of powers between the accumulation of evaluative evidence and the analysis of this evidence by paper evaluation functions (PEFs). PEFs can be freely defined by individuals or groups (e.g., scientific societies) and provide a plurality of perspectives on the scientific literature. Simple PEFs will use averages of ratings, weighting reviewers (e.g., by H-index), and rating scales (e.g., by relevance to a decision process) in different ways. Complex PEFs will use advanced statistical techniques to infer the quality of a paper. Papers with initially promising ratings will be more deeply evaluated. The continual refinement of PEFs in response to attempts by individuals to influence evaluations in their own favor will make the system ungameable. OA and OE together have the power to revolutionize scientific publishing and usher in a new culture of transparency, constructive criticism, and collaboration.”

URL : http://www.frontiersin.org/Computational_Neuroscience/10.3389/fncom.2012.00079/full

Cooperation between Referees and Authors Increases Peer Review…

Statut

Cooperation between Referees and Authors Increases Peer Review Accuracy :

“Peer review is fundamentally a cooperative process between scientists in a community who agree to review each other’s work in an unbiased fashion. Peer review is the foundation for decisions concerning publication in journals, awarding of grants, and academic promotion. Here we perform a laboratory study of open and closed peer review based on an online game. We show that when reviewer behavior was made public under open review, reviewers were rewarded for refereeing and formed significantly more cooperative interactions (13% increase in cooperation, P = 0.018). We also show that referees and authors who participated in cooperative interactions had an 11% higher reviewing accuracy rate (P = 0.016). Our results suggest that increasing cooperation in the peer review process can lead to a decreased risk of reviewing errors.”

URL : http://www.plosone.org/article/info:doi/10.1371/journal.pone.0026895

Open peer review by a selected papers network…

Posté le par

Open peer review by a selected-papers network :

“A selected-papers (SP) network is a network in which researchers who read, write, and review articles subscribe to each other based on common interests. Instead of reviewing a manuscript in secret for the Editor of a journal, each reviewer simply publishes his review (typically of a paper he wishes to recommend) to his SP network subscribers. Once the SP network reviewers complete their review decisions, the authors can invite any journal editor they want to consider these reviews and initial audience size, and make a publication decision. Since all impact assessment, reviews, and revisions are complete, this decision process should be short. I show how the SP network can provide a new way of measuring impact, catalyze the emergence of new subfields, and accelerate discovery in existing fields, by providing each reader a fine-grained filter for high-impact. I present a three phase plan for building a basic SP network, and making it an effective peer review platform that can be used by journals, conferences, users of repositories such as arXiv, and users of search engines such as PubMed. I show how the SP network can greatly improve review and dissemination of research articles in areas that are not well-supported by existing journals. Finally, I illustrate how the SP network concept can work well with existing publication services such as journals, conferences, arXiv, PubMed, and online citation management sites.”

URL : http://www.frontiersin.org/Journal/FullText.aspx?ART_DOI=10.3389/fncom.2012.00001&name=Computational_Neuroscience