Archives des mots-clés : Open peer review

On The Peer Review Reports: Does Size Matter?

Posté le par

Authors : Abdelghani Maddi, Luis Miotti

Amidst the ever-expanding realm of scientific production and the proliferation of predatory journals, the focus on peer review remains paramount for scientometricians and sociologists of science. Despite this attention, there is a notable scarcity of empirical investigations into the tangible impact of peer review on publication quality.

This study aims to address this gap by conducting a comprehensive analysis of how peer review contributes to the quality of scholarly publications, as measured by the citations they receive. Utilizing an adjusted dataset comprising 57,482 publications from Publons to Web of Science and employing the Raking Ratio method, our study reveals intriguing insights. Specifically, our findings shed light on a nuanced relationship between the length of reviewer reports and the subsequent citations received by publications.

Through a robust regression analysis, we establish that, beginning from 947 words, the length of reviewer reports is significantly associated with an increase in citations. These results not only confirm the initial hypothesis that longer reports indicate requested improvements, thereby enhancing the quality and visibility of articles, but also underscore the importance of timely and comprehensive reviewer reports.

Furthermore, insights from Publons’ data suggest that open access to reports can influence reviewer behavior, encouraging more detailed reports. Beyond the scholarly landscape, our findings prompt a reevaluation of the role of reviewers, emphasizing the need to recognize and value this resource-intensive yet underappreciated activity in institutional evaluations.

Additionally, the study sounds a cautionary note regarding the challenges faced by peer review in the context of an increasing volume of submissions, potentially compromising the vigilance of peers in swiftly assessing numerous articles.

HAL : https://cnrs.hal.science/hal-04492274

Additional experiments required: A scoping review of recent evidence on key aspects of Open Peer Review

Posté le par

Authors : Tony Ross-Hellauer, Serge P.J.M. Horbach

Diverse efforts are underway to reform the journal peer review system. Combined with growing interest in Open Science practices, Open Peer Review (OPR) has become of central concern to the scholarly community. However, what OPR is understood to encompass and how effective some of its elements are in meeting the expectations of diverse communities, are uncertain.

This scoping review updates previous efforts to summarize research on OPR to May 2022. Following the PRISMA methodological framework, it addresses the question: “What evidence has been reported in the scientific literature from 2017 to May 2022 regarding uptake, attitudes, and efficacy of two key aspects of OPR (Open Identities and Open Reports)?”

The review identifies, analyses and synthesizes 52 studies matching inclusion criteria, finding that OPR is growing, but still far from common practice. Our findings indicate positive attitudes towards Open Reports and more sceptical approaches to Open Identities.

Changes in reviewer behaviour seem limited and no evidence for lower acceptance rates of review invitations or slower turnaround times is reported in those studies examining those issues. Concerns about power dynamics and potential backfiring on critical reviews are in need of further experimentation.

We conclude with an overview of evidence gaps and suggestions for future research. Also, we discuss implications for policy and practice, both in the scholarly communications community and the research evaluation community more broadly.

URL : Additional experiments required: A scoping review of recent evidence on key aspects of Open Peer Review

DOI : https://doi.org/10.1093/reseval/rvae004

Attitudes toward open peer review among stakeholders of a scholar-led journal in Brazil

Posté le par

Authors : Leonardo Ferreira Fontenelle, Thiago Dias Sarti

Scholarly journals should consider the attitudes of their communities before adopting any of the seven traits of open peer review. Unfortunately, surveys from the Global North might not apply to the Global South, where double-blind peer review is commonplace even among natural sciences and medicine journals.

This paper reports the findings of a survey on attitudes toward open peer review among four stakeholder groups of a scholar-led medical journal in Brazil: society members, journal readers, authors, and reviewers.

Compared to a previous survey, which mostly recruited natural sciences researchers from Europe, this survey found similar support for open peer review in general and for most of its traits.

One important exception was open identities, which were considered detrimental by most participants, even more in this survey than in the previous one. Interestingly, participants were more dismissive of open identities as a whole than of statements about its specific consequences.

Because preprints are increasingly popular but incompatible with double-blind review, future research should examine the effects of transitioning from double-blind to open identities, especially on gender bias.

Meanwhile, scholarly journals with double-blind review might prefer to begin by adopting other traits of open review or to make open identities optional at first.

URL : Attitudes toward open peer review among stakeholders of a scholar-led journal in Brazil

DOI : https://doi.org/10.1590/2318-0889202133e200072

Modes d’évaluation ouverte par les pairs : de la revue à la plateforme

Posté le par

Auteurs/Authors : Evelyne Broudoux, Madjid Ihadjadene

Cet article a pour but de proposer un état de l’art des différentes formes de l’évaluation d’articles ou de communications par les pairs. De l’évaluation « aveugle» à l’évaluation « ouverte », de multiples possibilités existent et sont expérimentées.

C’est dans le champ des sciences que l’on trouve le plus d’innovations sociotechniques s’appuyant sur des plateformes de publication modélisant des workflows éditoriaux originaux.

L’ouverture de l’évaluation peut se produire entre pairs, en rendant publiques les identités et/ou les rapports des évaluateurs, à différents stades de l’article scientifique : préprint, en cours de rédaction, ou encore après publication.

Cet état de l’art est basé sur un ensemble de publications essentiellement produites par les acteurs de l’évaluation ouverte, issus principalement des disciplines STM.

URL : Modes d’évaluation ouverte par les pairs : de la revue à la plateforme

URL : https://revue-cossi.numerev.com/articles/revue-9/2496-modes-d-evaluation-ouverte-par-les-pairs-de-la-revue-a-la-plateforme

Open Research Data and Open Peer Review: Perceptions of a Medical and Health Sciences Community in Greece

Posté le par

Authors : Eirini Delikoura, Dimitrios Kouis

Recently significant initiatives have been launched for the dissemination of Open Access as part of the Open Science movement. Nevertheless, two other major pillars of Open Science such as Open Research Data (ORD) and Open Peer Review (OPR) are still in an early stage of development among the communities of researchers and stakeholders.

The present study sought to unveil the perceptions of a medical and health sciences community about these issues. Through the investigation of researchers‘ attitudes, valuable conclusions can be drawn, especially in the field of medicine and health sciences, where an explosive growth of scientific publishing exists.

A quantitative survey was conducted based on a structured questionnaire, with 179 valid responses. The participants in the survey agreed with the Open Peer Review principles. However, they ignored basic terms like FAIR (Findable, Accessible, Interoperable, and Reusable) and appeared incentivized to permit the exploitation of their data.

Regarding Open Peer Review (OPR), participants expressed their agreement, implying their support for a trustworthy evaluation system.

Conclusively, researchers need to receive proper training for both Open Research Data principles and Open Peer Review processes which combined with a reformed evaluation system will enable them to take full advantage of the opportunities that arise from the new scholarly publishing and communication landscape.

URL : Open Research Data and Open Peer Review: Perceptions of a Medical and Health Sciences Community in Greece

DOI : https://doi.org/10.3390/publications9020014

What Have We Learned from OpenReview?

Posté le par

Authors : Gang Wang, Qi Peng, Yanfeng Zhang, Mingyang Zhang

Anonymous peer review is used by the great majority of computer science conferences. OpenReview is such a platform that aims to promote openness in peer review process. The paper, (meta) reviews, rebuttals, and final decisions are all released to public. We collect 5,527 submissions and their 16,853 reviews from the OpenReview platform.

We also collect these submissions’ citation data from Google Scholar and their non-peer-reviewed versions from arXiv.org. By acquiring deep insights into these data, we have several interesting findings that could help understand the effectiveness of the public-accessible double-blind peer review process.

Our results can potentially help writing a paper, reviewing it, and deciding on its acceptance.

URL : https://arxiv.org/abs/2103.05885v4

The Pioneering Role of Sci in Post Publication Public Peer Review (P4R)

Posté le par

Authors : Ahmad Yaman Abdin, Muhammad Jawad Nasim, Yannick Ney, Claus Jacob

Scientists observe, discover, justify and eventually share their findings with the scientific community. Dissemination is an integral aspect of scientific discovery, since discoveries which go unnoticed have no or little impact on science.

Today, peer review is part of this process of scientific dissemination as it contributes proactively to the quality of a scientific article. As the numbers of scientific journals and scientific articles published therein are increasing steadily, processes such as the single-blind or double-blind peer review are facing a near collapse situation.

In fact, these traditional forms of reviewing have reached their limits and, because of this, are also increasingly considered as unfair, sloppy, superficial and even biased. In this manuscript, we propose forms of post-publication public peer review (P4R) as valuable alternatives to the traditional blind peer review system.

We describe how the journal Sci has explored such an approach and provide first empirical evidence of the benefits and also challenges, such a P4R approach faces.

URL : The Pioneering Role of Sci in Post Publication Public Peer Review (P4R)

DOI : https://doi.org/10.3390/publications9010013