Étiquette : Peer Review

Catégories
EN

What Have We Learned from OpenReview?

Authors : Gang Wang, Qi Peng, Yanfeng Zhang, Mingyang Zhang

Anonymous peer review is used by the great majority of computer science conferences. OpenReview is such a platform that aims to promote openness in peer review process. The paper, (meta) reviews, rebuttals, and final decisions are all released to public. We collect 5,527 submissions and their 16,853 reviews from the OpenReview platform.

We also collect these submissions’ citation data from Google Scholar and their non-peer-reviewed versions from arXiv.org. By acquiring deep insights into these data, we have several interesting findings that could help understand the effectiveness of the public-accessible double-blind peer review process.

Our results can potentially help writing a paper, reviewing it, and deciding on its acceptance.

URL : https://arxiv.org/abs/2103.05885v4

Catégories
EN

The Pioneering Role of Sci in Post Publication Public Peer Review (P4R)

Authors : Ahmad Yaman Abdin, Muhammad Jawad Nasim, Yannick Ney, Claus Jacob

Scientists observe, discover, justify and eventually share their findings with the scientific community. Dissemination is an integral aspect of scientific discovery, since discoveries which go unnoticed have no or little impact on science.

Today, peer review is part of this process of scientific dissemination as it contributes proactively to the quality of a scientific article. As the numbers of scientific journals and scientific articles published therein are increasing steadily, processes such as the single-blind or double-blind peer review are facing a near collapse situation.

In fact, these traditional forms of reviewing have reached their limits and, because of this, are also increasingly considered as unfair, sloppy, superficial and even biased. In this manuscript, we propose forms of post-publication public peer review (P4R) as valuable alternatives to the traditional blind peer review system.

We describe how the journal Sci has explored such an approach and provide first empirical evidence of the benefits and also challenges, such a P4R approach faces.

URL : The Pioneering Role of Sci in Post Publication Public Peer Review (P4R)

DOI : https://doi.org/10.3390/publications9010013

Catégories
EN

A Billion Dollar Donation: The Cost, and Inefficiency of, Researchers’ Time Spent on Peer Review

Authors : Balazs Aczel, Barnabas Szaszi, Alex Holcombe

Background

The amount and value of researchers’ peer review work is critical for academia and publishing. However, it is rarely recognized, its magnitude is unknown, and alternative ways of organizing peer review labor are rarely considered.

Methods

In this paper, we provide an estimate of researchers’ time and the salary-based contribution to the peer-review system, using publicly available data.

Results

We found that the total time reviewers globally worked on peer reviews was over 100 million hours in 2019, equivalent to over 12 thousand years. The estimated monetary value of the time US-based reviewers spent on reviews was over 1.1 billion USD in 2019. For China-based reviewers, the estimate is over 600 million USD, and for UK-based, over 200 million USD.

Conclusions

While these results are only rough estimates, they highlight the enormous amount of work and time that researchers provide to the publication system, and the importance of considering alternative ways of structuring, and paying for, peer review. We foster this process by discussing some alternative models that aim to improve the return on investment of scholarly publishing.

URL : A Billion Dollar Donation: The Cost, and Inefficiency of, Researchers’ Time Spent on Peer Review

DOI : https://doi.org/10.31222/osf.io/5h9z4

Catégories
EN

Ants-Review: A Privacy-Oriented Protocol for Incentivized Open Peer Reviews on Ethereum

Authors  : Bianca Trovò, Nazzareno Massari

Peer review is a necessary and essential quality control step for scientific publications but lacks proper incentives. Indeed, the process, which is very costly in terms of time and intellectual investment, not only is not remunerated by the journals but it is also not openly recognized by the academic community as a relevant scientific output for a researcher.

Therefore, scientific dissemination is affected in timeliness, quality and fairness. Here, to solve this issue, we propose a blockchain-based incentive system that rewards scientists for peer reviewing other scientists’ work and that builds up trust and reputation.

We designed a privacy-oriented protocol of smart contracts called Ants-Review that allows authors to issue a bounty for open anonymous peer reviews on Ethereum.

If requirements are met, peer reviews will be accepted and paid by the approver proportionally to their assessed quality. To promote ethical behaviour and inclusiveness the system implements a gamified mechanism that allows the whole community to evaluate the peer reviews and vote for the best ones.

DOI : https://doi.org/10.1007/978-3-030-71593-9_2

Catégories
EN

Journal policies and editors’ opinions on peer review

Authors : Daniel G Hamilton, Hannah Fraser, Rink Hoekstra, Fiona Fidler

Peer review practices differ substantially between journals and disciplines. This study presents the results of a survey of 322 editors of journals in ecology, economics, medicine, physics and psychology.

We found that 49% of the journals surveyed checked all manuscripts for plagiarism, that 61% allowed authors to recommend both for and against specific reviewers, and that less than 6% used a form of open peer review.

Most journals did not have an official policy on altering reports from reviewers, but 91% of editors identified at least one situation in which it was appropriate for an editor to alter a report. Editors were also asked for their views on five issues related to publication ethics.

A majority expressed support for co-reviewing, reviewers requesting access to data, reviewers recommending citations to their work, editors publishing in their own journals, and replication studies.

Our results provide a window into what is largely an opaque aspect of the scientific process. We hope the findings will inform the debate about the role and transparency of peer review in scholarly publishing.

URL : Journal policies and editors’ opinions on peer review

DOI : https://doi.org/10.7554/eLife.62529

Catégories
EN

The Enlightenment of Peer Review : How Academic Librarians Can Utilize Open Peer Review Methods to Advance Information Literacy

Author : Sandra Moore

In today’s world of digital scholarly publishing, it is increasingly clear that movements such as open access (OA), Open Science, and open peer review (OPR) are increasingly impactful and gaining momentum.

The shift towards openness in the academy reveals a transformation of traditional structures that compose scholarly communication as well as changing attitudes about the nature of authority and access within these systems.

These new directions in the scholarly information landscape have created a need for academic librarians to realign roles and respond in ways that build resiliency in an era of rapid change.

Recognizing that many core elements of scholarly communication are powerful tools for teaching students about information literacy can lead to transformative instructional strategies.

This paper explores how academic librarians can leverage the innovative traits of OPR to advance information literacy through experiential student learning opportunities grounded in the ACRL (2016) Framework for Information Literacy for Higher Education.

URL : The Enlightenment of Peer Review : How Academic Librarians Can Utilize Open Peer Review Methods to Advance Information Literacy

DOI : https://doi.org/10.5206/elip.v3i1.8618

Catégories
EN

Purposes of peer review: A qualitative study of stakeholder expectations and perceptions

Authors : Anna Severin, Joanna Chataway

Stakeholders might have diverging or conflicting expectations about the functions that peer review should fulfil. We aimed to explore how stakeholder groups perceive peer review and what they expect from it. We conducted qualitative focus group workshops with early‐, mid‐, and senior career scholars, editors, and publishers.

We recruited participants following a purposive maximum variation sampling approach. To identify purposes of peer review, we conducted a thematic analysis. Stakeholders expected peer review (1) to assess the contributions of a manuscript, (2) to conduct quality control, (3) to improve manuscripts, (4) to assess the suitability of manuscripts for a journal, (5) to provide a decision‐making tool for editors, (6) to provide feedback by peers, (7) to curate a community, and (8) to provide a seal of accreditation for published articles.

Stakeholders with different roles and tasks in the peer review process differed in the value they attached to the functions of peer review. Early‐ and mid‐career researchers valued social and feedback functions of peer review, while senior career researchers and editors expected it to instead perform a technical assessment of manuscripts and serve as a decision‐making tool.

Publishers expected peer review to assess the suitability of manuscripts for their journals and to provide a seal of accreditation. This revealed a potential tension between functions of peer review. Stakeholder expectations are shaped by how stakeholders perceive their own roles both in relation to the peer review process and within their scientific community.

URL : Purposes of peer review: A qualitative study of stakeholder expectations and perceptions

DOI : https://doi.org/10.1002/leap.1336