Emerging trends in peer review—a survey

Authors : Richard Walker, Pascal Rocha da Silva

“Classical peer review” has been subject to intense criticism for slowing down the publication process, bias against specific categories of paper and author, unreliability, inability to detect errors and fraud, unethical practices, and the lack of recognition for unpaid reviewers.

This paper surveys innovative forms of peer review that attempt to address these issues. Based on an initial literature review, we construct a sample of 82 channels of scientific communication covering all forms of review identified by the survey, and analyze the review mechanisms used by each channel.

We identify two major trends: the rapidly expanding role of preprint servers (e.g., ArXiv) that dispense with traditional peer review altogether, and the growth of “non-selective review,” focusing on papers’ scientific quality rather than their perceived importance and novelty.

Other potentially important developments include forms of “open review,” which remove reviewer anonymity, and interactive review, as well as new mechanisms for post-publication review and out-of-channel reader commentary, especially critical commentary targeting high profile papers.

One of the strongest findings of the survey is the persistence of major differences between the peer review processes used by different disciplines. None of these differences is likely to disappear in the foreseeable future.

The most likely scenario for the coming years is thus continued diversification, in which different review mechanisms serve different author, reader, and publisher needs. Relatively little is known about the impact of these innovations on the problems they address. These are important questions for future quantitative research.

URL : Emerging trends in peer review—a survey

DOI : https://doi.org/10.3389/fnins.2015.00169

Opening Scholarly Communication in Social Sciences by Connecting Collaborative Authoring to Peer Review

Authors : Afshin Sadeghi, Johannes Wilm, Philipp Mayr, Christoph Lange

The objective of the OSCOSS research project on « Opening Scholarly Communication in the Social Sciences » is to build a coherent collaboration environment that facilitates scholarly communication workflows of social scientists in the roles of authors, reviewers, editors and readers. This paper presents the implementation of the core of this environment: the integration of the Fidus Writer academic word processor with the Open Journal Systems (OJS) submission and review management system.

URL : https://arxiv.org/abs/1703.04428

The Global Burden of Journal Peer Review in the Biomedical Literature: Strong Imbalance in the Collective Enterprise

Authors : Michail Kovanis, Raphaël Porcher, Philippe Ravaud, Ludovic Trinquart

The growth in scientific production may threaten the capacity for the scientific community to handle the ever-increasing demand for peer review of scientific publications. There is little evidence regarding the sustainability of the peer-review system and how the scientific community copes with the burden it poses.

We used mathematical modeling to estimate the overall quantitative annual demand for peer review and the supply in biomedical research. The modeling was informed by empirical data from various sources in the biomedical domain, including all articles indexed at MEDLINE.

We found that for 2015, across a range of scenarios, the supply exceeded by 15% to 249% the demand for reviewers and reviews. However, 20% of the researchers performed 69% to 94% of the reviews.

Among researchers actually contributing to peer review, 70% dedicated 1% or less of their research work-time to peer review while 5% dedicated 13% or more of it. An estimated 63.4 million hours were devoted to peer review in 2015, among which 18.9 million hours were provided by the top 5% contributing reviewers.

Our results support that the system is sustainable in terms of volume but emphasizes a considerable imbalance in the distribution of the peer-review effort across the scientific community.

Finally, various individual interactions between authors, editors and reviewers may reduce to some extent the number of reviewers who are available to editors at any point.

URL : The Global Burden of Journal Peer Review in the Biomedical Literature: Strong Imbalance in the Collective Enterprise

DOI : http://dx.doi.org/10.1371/journal.pone.0166387

Day of the week effect in paper submission/acceptance/rejection to/in/by peer review journals. II. An ARCH econometric-like modeling

Authors : Marcel Ausloos, Olgica Nedic, Aleksandar Dekanski, Maciej J. Mrowinski, Piotr Fronczak, Agata Fronczak

This paper aims at providing a statistical model for the preferred behavior of authors submitting a paper to a scientific journal. The electronic submission of (about 600) papers to the Journal of the Serbian Chemical Society has been recorded for every day from Jan. 01, 2013 till Dec. 31, 2014, together with the acceptance or rejection paper fate.

Seasonal effects and editor roles (through desk rejection and subfield editors) are examined. An ARCH-like econometric model is derived stressing the main determinants of the favorite day-of-week process.

URL : https://arxiv.org/abs/1611.04639

Open Post Publication Peer Review: An Innovation in Scientific Communication

Authors : Peiling Wang, Manasa Rath, Michael Deike, Wu Qiang

This research observes the emerging open peer review journals. In scientific publishing, transparency in peer review is a growing topic of interest for online journals. The traditional blind refereeing process has been criticized for lacking transparency.

Although the idea of open peer review (OPR) has been explored since 1980s, it is only in this decade that OPR journals are born. Towards a more open publishing model, the peer review process–once accessible only to the editors and referees—is now available to public.

The published article and its review history are being integrated into one entity; readers can submit or post comments to extend the peer process. This preliminary study observed four pioneer OPR journals representing pre-publication OPR and post-publication OPR.

Data collection focuses on publication’s lifecycle from its submission to peer approval. Preliminary results include comparisons of the level of openness and nature of interactions during refereeing process.

URL : http://trace.tennessee.edu/utk_infosciepubs/55/

Opening Review in LIS Journals: A Status Report

Author : Emily Ford

INTRODUCTION

Peer-review practices in scholarly publishing are changing. Digital publishing mechanisms allow for open peer review, a peer review process that discloses author and reviewer identities to one another.

This model of peer review is increasingly implemented in scholarly publishing. In science, technology, engineering, and math (STEM) disciplines, open peer review is implemented in journal publishing processes, and, in the humanities and social sciences, it is often coupled with new scholarship practices, such as the digital humanities.

This article reports findings from an exploratory study on peer-review and publishing practices in Library and Information Science (LIS), focusing on LIS’s relationships with open peer review.

METHODS

Editors of LIS journals were surveyed regarding journal peer review and publishing practices.

RESULTS

This article reports the general “pulse” of attitudes and conversations regarding open peer review and discusses its challenges in LIS. Results show an ideological split between traditionally-published journals and open access and association-affiliated journals. Open access and association-affiliated journal editors are more likely to consider investigating open peer review.

DISCUSSION

The LIS community of journal editors, authors, reviewers, and readers need to discuss open peer review as well as experiment with it. Experiments with open peer review in scholarly LIS publishing will inform our praxis as librarians.

URL : Opening Review in LIS Journals: A Status Report

DOI : http://doi.org/10.7710/2162-3309.2148

The Miracle of Peer Review and Development in Science: An Agent-Based Model

Authors : Simone Righi, Károly Takács

It is not easy to rationalize how peer review, as the current grassroots of science, can work based on voluntary contributions of reviewers. There is no rationale to write impartial and thorough evaluations.

Consequently, there is no risk in submitting low-quality work by authors. As a result, scientists face a social dilemma: if everyone acts according to his or her own self-interest, low scientific quality is produced. Still, in practice, reviewers as well as authors invest high effort in reviews and submissions.

We examine how the increased relevance of public good benefits (journal impact factor), the editorial policy of handling incoming reviews, and the acceptance decisions that take into account reputational information can help the evolution of high-quality contributions from authors.

High effort from the side of reviewers is problematic even if authors cooperate: reviewers are still best off by producing low-quality reviews, which does not hinder scientific development, just adds random noise and unnecessary costs to it.

We show with agent-based simulations that tacit agreements between authors that are based on reciprocity might decrease these costs, but does not result in superior scientific quality. Our study underlines why certain self-emerged current practices, such as the increased importance of journal metrics, the reputation-based selection of reviewers, and the reputation bias in acceptance work efficiently for scientific development.

Our results find no answers, however, how the system of peer review with impartial and thorough evaluations could be sustainable jointly with rapid scientific development.

URL : http://arxiv.org/abs/1607.02574