Archives des mots-clés : Peer Review

The Enlightenment of Peer Review : How Academic Librarians Can Utilize Open Peer Review Methods to Advance Information Literacy

Posté le par

Author : Sandra Moore

In today’s world of digital scholarly publishing, it is increasingly clear that movements such as open access (OA), Open Science, and open peer review (OPR) are increasingly impactful and gaining momentum.

The shift towards openness in the academy reveals a transformation of traditional structures that compose scholarly communication as well as changing attitudes about the nature of authority and access within these systems.

These new directions in the scholarly information landscape have created a need for academic librarians to realign roles and respond in ways that build resiliency in an era of rapid change.

Recognizing that many core elements of scholarly communication are powerful tools for teaching students about information literacy can lead to transformative instructional strategies.

This paper explores how academic librarians can leverage the innovative traits of OPR to advance information literacy through experiential student learning opportunities grounded in the ACRL (2016) Framework for Information Literacy for Higher Education.

URL : The Enlightenment of Peer Review : How Academic Librarians Can Utilize Open Peer Review Methods to Advance Information Literacy

DOI : https://doi.org/10.5206/elip.v3i1.8618

Purposes of peer review: A qualitative study of stakeholder expectations and perceptions

Posté le par

Authors : Anna Severin, Joanna Chataway

Stakeholders might have diverging or conflicting expectations about the functions that peer review should fulfil. We aimed to explore how stakeholder groups perceive peer review and what they expect from it. We conducted qualitative focus group workshops with early‐, mid‐, and senior career scholars, editors, and publishers.

We recruited participants following a purposive maximum variation sampling approach. To identify purposes of peer review, we conducted a thematic analysis. Stakeholders expected peer review (1) to assess the contributions of a manuscript, (2) to conduct quality control, (3) to improve manuscripts, (4) to assess the suitability of manuscripts for a journal, (5) to provide a decision‐making tool for editors, (6) to provide feedback by peers, (7) to curate a community, and (8) to provide a seal of accreditation for published articles.

Stakeholders with different roles and tasks in the peer review process differed in the value they attached to the functions of peer review. Early‐ and mid‐career researchers valued social and feedback functions of peer review, while senior career researchers and editors expected it to instead perform a technical assessment of manuscripts and serve as a decision‐making tool.

Publishers expected peer review to assess the suitability of manuscripts for their journals and to provide a seal of accreditation. This revealed a potential tension between functions of peer review. Stakeholder expectations are shaped by how stakeholders perceive their own roles both in relation to the peer review process and within their scientific community.

URL : Purposes of peer review: A qualitative study of stakeholder expectations and perceptions

DOI : https://doi.org/10.1002/leap.1336

Revealing Reviewers’ Identities as Part of Open Peer Review and Analysis of the Review Reports

Posté le par

Authors :  Cezary Bolek, Dejan Marolov, Monika Bolek, Jovan Shopovski

This research article is aimed at comparing review reports, in which the identity of the reviewers is revealed to the authors of the papers, with those where the reviewers decided to remain anonymous.

The review reports are gathered as part of the peer review process of the European Scientific Journal (ESJ). This journal maintains a single-blind peer review procedure and optional open review. Reviewers are familiar with the names of the authors but not vice versa. When sending the review reports, the reviewers can opt to reveal their identity to the authors.

The sample of 343 review reports from members of the ESJ editorial board, gathered within the period of May to July 2019, were analysed. The data analysis was performed using the Python programing language based on NumPy, Pandas, and Scipy packages.

Half of the reviewers decided to choose the open option and reveal their names to the authors of the papers. The other half remained anonymous. The results show that female reviewers more often decide to remain anonymous than their male colleagues. However, there is no significant difference in the review reports on the basis of gender or country of institutional affiliation of the reviewers.

Revealing identities did not make a difference in the reviewers’ point appraisal in the review reports. This difference was not significant. However, a majority of the reviewers who recommended rejection in their review reports were not willing to reveal their identities.

Even more, those reviewers who revealed their identity were more likely to recommend in their review reports acceptance without revision or a minor revision.

URL : Revealing Reviewers’ Identities as Part of Open Peer Review and Analysis of the Review Reports

DOI : http://doi.org/10.18352/lq.10347

PRINCIPIA: a Decentralized Peer-Review Ecosystem

Posté le par

Authors : Andrea Mambrini, Andrea Baronchelli, Michele Starnini, Daniele Marinazzo, Manlio De Domenico

Peer review is a cornerstone of modern scientific endeavor. However, there is growing consensus that several limitations of the current peer review system, from lack of incentives to reviewers to lack of transparency, risks to undermine its benefits.

Here, we introduce the PRINCIPIA (http://www.principia.network/) framework for peer-review of scientific outputs (e.g., papers, grant proposals or patents).

The framework allows key players of the scientific ecosystem — including existing publishing groups — to create and manage peer-reviewed journals, by building a free market for reviews and publications. PRINCIPIA’s referees are transparently rewarded according to their efforts and the quality of their reviews.

PRINCIPIA also naturally allows to recognize the prestige of users and journals, with an intrinsic reputation system that does not depend on third-parties. PRINCIPIA re-balances the power between researchers and publishers, stimulates valuable assessments from referees, favors a fair competition between journals, and reduces the costs to access research output and to publish.

URL : https://arxiv.org/abs/2008.09011

Quantifying professionalism in peer review

Posté le par

Authors : Travis G. Gerwing, Alyssa M. Allen Gerwing, Stephanie Avery-Gomm, Chi-Yeung Choi, Jeff C. Clements, Joshua A. Rash

Background

The process of peer-review in academia has attracted criticism surrounding issues of bias, fairness, and professionalism; however, frequency of occurrence of such comments is unknown.

Methods

We evaluated 1491 sets of reviewer comments from the fields of “Ecology and Evolution” and “Behavioural Medicine,” of which 920 were retrieved from the online review repository Publons and 571 were obtained from six early career investigators.

Comment sets were coded for the occurrence of “unprofessional comments” and “incomplete, inaccurate or unsubstantiated critiques” using an a-prior rubric based on our published research. Results are presented as absolute numbers and percentages.

Results

Overall, 12% (179) of comment sets included at least one unprofessional comment towards the author or their work, and 41% (611) contained incomplete, inaccurate of unsubstantiated critiques (IIUC).

Conclusions

The large number of unprofessional comments, and IIUCs observed could heighten psychological distress among investigators, particularly those at an early stage in their career. We suggest that development and adherence to a universally agreed upon reviewer code of conduct is necessary to improve the quality and professional experience of peer review.

URL : Quantifying professionalism in peer review

DOI : https://doi.org/10.1186/s41073-020-00096-x

Open peer review: promoting transparency in open science

Posté le par

Authors : Dietmar Wolfram, Peiling Wang, Adam Hembree, Hyoungjoo Park

Open peer review (OPR), where review reports and reviewers’ identities are published alongside the articles, represents one of the last aspects of the open science movement to be widely embraced, although its adoption has been growing since the turn of the century.

This study provides the first comprehensive investigation of OPR adoption, its early adopters and the implementation approaches used. Current bibliographic databases do not systematically index OPR journals, nor do the OPR journals clearly state their policies on open identities and open reports.

Using various methods, we identified 617 OPR journals that published at least one article with open identities or open reports as of 2019 and analyzed their wide-ranging implementations to derive emerging OPR practices.

The findings suggest that: (1) there has been a steady growth in OPR adoption since 2001, when 38 journals initially adopted OPR, with more rapid growth since 2017; (2) OPR adoption is most prevalent in medical and scientific disciplines (79.9%); (3) five publishers are responsible for 81% of the identified OPR journals; (4) early adopter publishers have implemented OPR in different ways, resulting in different levels of transparency.

Across the variations in OPR implementations, two important factors define the degree of transparency: open identities and open reports. Open identities may include reviewer names and affiliation as well as credentials; open reports may include timestamped review histories consisting of referee reports and author rebuttals or a letter from the editor integrating reviewers’ comments.

When and where open reports can be accessed are also important factors indicating the OPR transparency level. Publishers of optional OPR journals should add metric data in their annual status reports.

URL : Open peer review: promoting transparency in open science

DOI : https://doi.org/10.1007/s11192-020-03488-4

Innovative Strategies for Peer Review

Posté le par

Author : Edward Barroga

Peer review is a crucial part of research and publishing. However, it remains imperfect and suffers from bias, lack of transparency, and professional jealousy. It is also overburdened by an increasing quantity of complex papers against the stagnant pool of reviewers, causing delays in peer review.

Additionally, many medical, nursing, and healthcare educators, peer reviewers, and authors may not be completely familiar with the current changes in peer review. Moreover, reviewer education and training have unfortunately remained lacking.

This is especially crucial since current initiatives to improve the review process are now influenced by factors other than academic needs. Thus, increasing attention has recently focused on ways of streamlining the peer review process and implementing alternative peer-review methods using new technologies and open access models.

This article aims to give an overview of the innovative strategies for peer review and to consider perspectives that may be helpful in introducing changes to peer review. Critical assessments of peer review innovations and incentives based on past and present experiences are indispensable.

A theoretical appraisal must be balanced by a realistic appraisal of the ethical roles of all stakeholders in enhancing the peer review process.

As the peer review system is far from being perfect, identifying and developing core competencies among reviewers, continuing education of researchers, reviewer education and training, and professional engagement of the scientific community in various disciplines may help bridge gaps in an imperfect but indispensable peer review system.

URL : Innovative Strategies for Peer Review

DOI : https://doi.org/10.3346/jkms.2020.35.e138