Archives des mots-clés : Peer Review

What is open peer review? A systematic review

Posté le par

Author : Tony Ross-Hellauer

Background

“Open peer review” (OPR), despite being a major pillar of Open Science, has neither a standardized definition nor an agreed schema of its features and implementations. The literature reflects this, with a myriad of overlapping and often contradictory definitions.

While the term is used by some to refer to peer review where the identities of both author and reviewer are disclosed to each other, for others it signifies systems where reviewer reports are published alongside articles.

For others it signifies both of these conditions, and for yet others it describes systems where not only “invited experts” are able to comment. For still others, it includes a variety of combinations of these and other novel methods.

Methods

Recognising the absence of a consensus view on what open peer review is, this article undertakes a systematic review of definitions of “open peer review” or “open review”, to create a corpus of 122 definitions.

These definitions are then systematically analysed to build a coherent typology of the many different innovations in peer review signified by the term, and hence provide the precise technical definition currently lacking.

Results

This quantifiable data yields rich information on the range and extent of differing definitions over time and by broad subject area. Quantifying definitions in this way allows us to accurately portray exactly how  ambiguously the phrase “open peer review”  has been used thus far, for the literature offers a total of 22 distinct configurations of seven traits, effectively meaning that there are 22 different definitions of OPR in the literature.

Conclusions

Based on this work, I propose a pragmatic definition of open peer review as an umbrella term for a number of overlapping ways that peer review models can be adapted in line with the ethos of Open Science, including making reviewer and author identities open, publishing review reports and enabling greater participation in the peer review process.

URL : What is open peer review? A systematic review

DOI : http://dx.doi.org/10.12688/f1000research.11369.1

What Constitutes Peer Review of Data: A survey of published peer review guidelines

Posté le par

Author : Todd A Carpenter

Since a number of journals specifically focus on the review and publication of data sets, reviewing their policies seems an appropriate place to start in assessing what existing practice looks like in the ‘real world’ of reviewing and publishing data.

This article outlines a study of the publicly available peer review policies of 39 scientific publications that publish data papers to discern which criteria are most and least frequently referenced. It also compares current practice with proposed criteria published in 2012.

URL : What Constitutes Peer Review of Data: A survey of published peer review guidelines

Alternative location : https://arxiv.org/abs/1704.02236

Emerging trends in peer review—a survey

Posté le par

Authors : Richard Walker, Pascal Rocha da Silva

“Classical peer review” has been subject to intense criticism for slowing down the publication process, bias against specific categories of paper and author, unreliability, inability to detect errors and fraud, unethical practices, and the lack of recognition for unpaid reviewers.

This paper surveys innovative forms of peer review that attempt to address these issues. Based on an initial literature review, we construct a sample of 82 channels of scientific communication covering all forms of review identified by the survey, and analyze the review mechanisms used by each channel.

We identify two major trends: the rapidly expanding role of preprint servers (e.g., ArXiv) that dispense with traditional peer review altogether, and the growth of “non-selective review,” focusing on papers’ scientific quality rather than their perceived importance and novelty.

Other potentially important developments include forms of “open review,” which remove reviewer anonymity, and interactive review, as well as new mechanisms for post-publication review and out-of-channel reader commentary, especially critical commentary targeting high profile papers.

One of the strongest findings of the survey is the persistence of major differences between the peer review processes used by different disciplines. None of these differences is likely to disappear in the foreseeable future.

The most likely scenario for the coming years is thus continued diversification, in which different review mechanisms serve different author, reader, and publisher needs. Relatively little is known about the impact of these innovations on the problems they address. These are important questions for future quantitative research.

URL : Emerging trends in peer review—a survey

DOI : https://doi.org/10.3389/fnins.2015.00169

Opening Scholarly Communication in Social Sciences by Connecting Collaborative Authoring to Peer Review

Posté le par

Authors : Afshin Sadeghi, Johannes Wilm, Philipp Mayr, Christoph Lange

The objective of the OSCOSS research project on “Opening Scholarly Communication in the Social Sciences” is to build a coherent collaboration environment that facilitates scholarly communication workflows of social scientists in the roles of authors, reviewers, editors and readers. This paper presents the implementation of the core of this environment: the integration of the Fidus Writer academic word processor with the Open Journal Systems (OJS) submission and review management system.

URL : https://arxiv.org/abs/1703.04428

The Global Burden of Journal Peer Review in the Biomedical Literature: Strong Imbalance in the Collective Enterprise

Posté le par

Authors : Michail Kovanis, Raphaël Porcher, Philippe Ravaud, Ludovic Trinquart

The growth in scientific production may threaten the capacity for the scientific community to handle the ever-increasing demand for peer review of scientific publications. There is little evidence regarding the sustainability of the peer-review system and how the scientific community copes with the burden it poses.

We used mathematical modeling to estimate the overall quantitative annual demand for peer review and the supply in biomedical research. The modeling was informed by empirical data from various sources in the biomedical domain, including all articles indexed at MEDLINE.

We found that for 2015, across a range of scenarios, the supply exceeded by 15% to 249% the demand for reviewers and reviews. However, 20% of the researchers performed 69% to 94% of the reviews.

Among researchers actually contributing to peer review, 70% dedicated 1% or less of their research work-time to peer review while 5% dedicated 13% or more of it. An estimated 63.4 million hours were devoted to peer review in 2015, among which 18.9 million hours were provided by the top 5% contributing reviewers.

Our results support that the system is sustainable in terms of volume but emphasizes a considerable imbalance in the distribution of the peer-review effort across the scientific community.

Finally, various individual interactions between authors, editors and reviewers may reduce to some extent the number of reviewers who are available to editors at any point.

URL : The Global Burden of Journal Peer Review in the Biomedical Literature: Strong Imbalance in the Collective Enterprise

DOI : http://dx.doi.org/10.1371/journal.pone.0166387

Day of the week effect in paper submission/acceptance/rejection to/in/by peer review journals. II. An ARCH econometric-like modeling

Posté le par

Authors : Marcel Ausloos, Olgica Nedic, Aleksandar Dekanski, Maciej J. Mrowinski, Piotr Fronczak, Agata Fronczak

This paper aims at providing a statistical model for the preferred behavior of authors submitting a paper to a scientific journal. The electronic submission of (about 600) papers to the Journal of the Serbian Chemical Society has been recorded for every day from Jan. 01, 2013 till Dec. 31, 2014, together with the acceptance or rejection paper fate.

Seasonal effects and editor roles (through desk rejection and subfield editors) are examined. An ARCH-like econometric model is derived stressing the main determinants of the favorite day-of-week process.

URL : https://arxiv.org/abs/1611.04639

Open Post Publication Peer Review: An Innovation in Scientific Communication

Posté le par

Authors : Peiling Wang, Manasa Rath, Michael Deike, Wu Qiang

This research observes the emerging open peer review journals. In scientific publishing, transparency in peer review is a growing topic of interest for online journals. The traditional blind refereeing process has been criticized for lacking transparency.

Although the idea of open peer review (OPR) has been explored since 1980s, it is only in this decade that OPR journals are born. Towards a more open publishing model, the peer review process–once accessible only to the editors and referees—is now available to public.

The published article and its review history are being integrated into one entity; readers can submit or post comments to extend the peer process. This preliminary study observed four pioneer OPR journals representing pre-publication OPR and post-publication OPR.

Data collection focuses on publication’s lifecycle from its submission to peer approval. Preliminary results include comparisons of the level of openness and nature of interactions during refereeing process.

URL : http://trace.tennessee.edu/utk_infosciepubs/55/