Guidelines for open peer review implementation

Authors : Tony Ross-Hellauer, Edit Görögh

Open peer review (OPR) is moving into the mainstream, but it is often poorly understood and surveys of researcher attitudes show important barriers to implementation.

As more journals move to implement and experiment with the myriad of innovations covered by this term, there is a clear need for best practice guidelines to guide implementation.

This brief article aims to address this knowledge gap, reporting work based on an interactive stakeholder workshop to create best-practice guidelines for editors and journals who wish to transition to OPR.

Although the advice is aimed mainly at editors and publishers of scientific journals, since this is the area in which OPR is at its most mature, many of the principles may also be applicable for the implementation of OPR in other areas (e.g., books, conference submissions).

URL : Guidelines for open peer review implementation

DOI : https://doi.org/10.1186/s41073-019-0063-9

The effect of publishing peer review reports on referee behavior in five scholarly journals

Authors : Giangiacomo Bravo, Francisco Grimaldo, Emilia López-Iñesta, Bahar Mehmani, Flaminio Squazzoni

To increase transparency in science, some scholarly journals are publishing peer review reports. But it is unclear how this practice affects the peer review process. Here, we examine the effect of publishing peer review reports on referee behavior in five scholarly journals involved in a pilot study at Elsevier.

By considering 9,220 submissions and 18,525 reviews from 2010 to 2017, we measured changes both before and during the pilot and found that publishing reports did not significantly compromise referees’ willingness to review, recommendations, or turn-around times.

Younger and non-academic scholars were more willing to accept to review and provided more positive and objective recommendations. Male referees tended to write more constructive reports during the pilot.

Only 8.1% of referees agreed to reveal their identity in the published report. These findings suggest that open peer review does not compromise the process, at least when referees are able to protect their anonymity.

URL : The effect of publishing peer review reports on referee behavior in five scholarly journals

DOI : https://doi.org/10.1038/s41467-018-08250-2

Automatically Annotating Articles Towards Opening and Reusing Transparent Peer Reviews

Authors : Afshin Sadeghi, Sarven Capadisli, Johannes Wilm, Christoph Lange, Philipp Mayr

An increasing number of scientific publications are created in open and transparent peer review models: a submission is published first, and then reviewers are invited, or a submission is reviewed in a closed environment but then these reviews are published with the final article, or combinations of these.

Reasons for open peer review include giving better credit to reviewers and enabling readers to better appraise the quality of a publication. In most cases, the full, unstructured text of an open review is published next to the full, unstructured text of the article reviewed.

This approach prevents human readers from getting a quick impression of the quality of parts of an article, and it does not easily support secondary exploitation, e.g., for scientometrics on reviews.

While document formats have been proposed for publishing structured articles including reviews, integrated tool support for entire open peer review workflows resulting in such documents is still scarce.

We present AR-Annotator, the Automatic Article and Review Annotator which employs a semantic information model of an article and its reviews, using semantic markup and unique identifiers for all entities of interest.

The fine-grained article structure is not only exposed to authors and reviewers but also preserved in the published version. We publish articles and their reviews in a Linked Data representation and thus maximize their reusability by third-party applications.

We demonstrate this reusability by running quality-related queries against the structured representation of articles and their reviews.

URL : https://arxiv.org/abs/1812.01027

Reviewing the review process: Investigation of researchers’ opinions on different methods of peer review

Author : Carolin Anna Rebernig

Peer review is considered the gold standard of scientific publishing. Trust in the traditional system of editor – blind-reviewer – author is still high, but it’s authority is in decline and alternative methods are on the rise.

The current study investigates opinions of alternative peer review methods, the arguments for and against, and the reasons why academics are searching for new approaches. The opinions were analysed by applying qualitative content analysis to online discussions.

The findings were interpreted using two different sociological theories: the Mertonian sociology of science and social constructivism.

The results of the study show that the most discussed method was also the most traditional one: closed pre-publication peer review comprised of single blind, double-blind and open peer review (non blinded).

Discussions of open peer review (both open publishing of reports and open discussions) were also common. All other alternative methods were discussed much less. But the discussions were lively and each method was discussed in both positive and negative terms.

The reasons for preferring certain methods were also manifold, but dominant topics were bias and fairness, quality issues (regarding reviews and publications), issues concerning human resources and communication and exchange among people.

The results of this study demonstrate that while ethical norms seems to be a scientific ideal, human nature makes it impossible to accomplish this goal.

URN : http://urn.kb.se/resolve?urn=urn%3Anbn%3Ase%3Ahb%3Adiva-14607

Scholarship as an Open Conversation: Utilizing Open Peer Review in Information Literacy Instruction

Author : Emily Ford

This article explores the ACRL Framework for Information Literacy’s frame, Scholarship as a Conversation. This frame asserts that information literate students have the disposition, skills, and knowledge to recognize and participate in disciplinary scholarly conversations.

By investigating the peer-review process as part of scholarly conversations, this article provides a brief literature review on peer review in information literacy instruction, and argues that by using open peer review (OPR) models for teaching, library workers can allow students to gain a deeper understanding of scholarly conversations.

OPR affords students the ability to begin dismantling the systemic oppression that blinded peer review and the traditional scholarly publishing system reinforce. Finally, the article offers an example classroom activity using OPR to help students enter scholarly conversations, and recognize power and oppression in scholarly publishing.

URL : http://www.inthelibrarywiththeleadpipe.org/2018/open-conversation/

Ten considerations for open peer review

Authors : Birgit Schmidt, Tony Ross-Hellauer, Xenia van Edig, Elizabeth C Moylan

Open peer review (OPR), as with other elements of open science and open research, is on the rise. It aims to bring greater transparency and participation to formal and informal peer review processes.

But what is meant by `open peer review’, and what advantages and disadvantages does it have over standard forms of review? How do authors or reviewers approach OPR? And what pitfalls and opportunities should you look out for?

Here, we propose ten considerations for OPR, drawing on discussions with authors, reviewers, editors, publishers and librarians, and provide a pragmatic, hands-on introduction to these issues.

We cover basic principles and summarise best practices, indicating how to use OPR to achieve best value and mutual benefits for all stakeholders and the wider research community.

URL : Ten considerations for open peer review

DOI : http://dx.doi.org/10.12688/f1000research.15334.1

Developing indicators on Open Access by combining evidence from diverse data sources

Authors : Thed van Leeuwen, Ingeborg Meijer, Alfredo Yegros-Yegros, Rodrigo Costas

In the last couple of years, the role of Open Access (OA) publishing has become central in science management and research policy. In the UK and the Netherlands, national OA mandates require the scientific community to seriously consider publishing research outputs in OA forms.

At the same time, other elements of Open Science are becoming also part of the debate, thus including not only publishing research outputs but also other related aspects of the chain of scientific knowledge production such as open peer review and open data.

From a research management point of view, it is important to keep track of the progress made in the OA publishing debate. Until now, this has been quite problematic, given the fact that OA as a topic is hard to grasp by bibliometric methods, as most databases supporting bibliometric data lack exhaustive and accurate open access labelling of scientific publications.

In this study, we present a methodology that systematically creates OA labels for large sets of publications processed in the Web of Science database. The methodology is based on the combination of diverse data sources that provide evidence of publications being OA.

URL : https://arxiv.org/abs/1802.02827v1