Étiquette : Peer Review

Catégories
EN

Effectiveness of the researcher-led “Peerspectives” peer review training course on review quality, knowledge, and skills among doctoral students in the biomedical sciences: a pre-post study

Authors : Jessica L. Rohmann, Nadja Wülk, Kerstin Rubarth, Hannah Grillmaier, Iman Abdikarim, Mariana Lopes Simões, Sara Schroter, Marco Piccininni, Tobias Kurth, Toivo Glatz

Background

Peer review remains a cornerstone of scientific knowledge dissemination, yet comprehensive, practically relevant training is limited. This inspired us to develop Peerspectives, a peer review training course for doctoral students in the biomedical sciences in Berlin, Germany. We aimed to assess the effectiveness of the Peerspectives course on editor-judged quality of peer review reports.

Methods

Doctoral students in health research fields who enrolled in the Peerspectives course between October 2020 and August 2022 were invited to participate in the study, and 80 consented. The ~18 week-long course provided training on the structure, purpose, and conduct of peer review and editorial processes in biomedical journals. It included 12 h of lectures, homework assignments, and 12 h of hands-on, small-group workshops, during which students reviewed original research manuscripts currently under consideration at The BMJ under the guidance of experienced mentors.

The primary outcome was the overall quality of the peer review reports as judged by two independent BMJ editors using the global score of the Review Quality Instrument (RQI) pre- and post-intervention. Additionally, we compared participants’ post-course scores with those of actual BMJ reviewers. We also compared participants’ self-assessed knowledge and skills related to scholarly peer review (1–5 Likert scale) before and after the course.

Results

After course completion, the editor-assessed overall quality of the participants’ peer review reports was higher than before the course (median increase of 0.5 points, p < 0.001; mean increase of 0.36 points, p < 0.001). The RQI scores of participants’ post-course reports were not non-inferior to those of actual BMJ reviewers for the same manuscripts. Self-assessed peer review-related knowledge skills increased across all questionnaire items after course completion. Greatest improvements were seen in understanding reviewer expectations (increase in means from 2.9 to 4.5), confidence in reviewing (2.5 to 3.9), and knowing what to look for while reviewing (2.8 to 4.2).

Conclusions

Providing doctoral students with comprehensive training resulted in an editorially significant increase in review report quality and improved understanding of the role and expectations of peer reviewers in the scholarly publishing processes and confidence in giving constructive feedback.

URL : Effectiveness of the researcher-led “Peerspectives” peer review training course on review quality, knowledge, and skills among doctoral students in the biomedical sciences: a pre-post study

DOI : https://doi.org/10.1186/s41073-026-00220-3

Catégories
EN

Un/Sustainable Peer Review and Generative AI: Ethical Gaps, Editorial Acceleration, and the Whitewashing of Technological Solutionism

Authors : Angel Gord, Chris H. Gray, Ana Rodrígue, Elías Said-Hung, Raúl Tabaré

Generative AI in peer review raises ethical and environmental concerns and risks deepening existing inequities in scholarly publishing. Celebrated gains in speed often mask declines in quality and accountability.

Training and deploying large models impose environmental costs. In editorial workflows, AI can privilege technical fixes over structural reform, and evidence shows it reproduces human biases while being cast as neutral. We call for a renewed commitment to open-science principles anchored in human oversight, deep sustainability, and broader justice.

The paper concludes by interrogating sustainability’s absence from green-economy debates and mapping the values likely to shape the future of peer review.

URL : Un/Sustainable Peer Review and Generative AI: Ethical Gaps, Editorial Acceleration, and the Whitewashing of Technological Solutionism

DOI : https://doi.org/10.17742/IMAGE29731

Catégories
EN

Determining quality dimensions for peer review reports using a Delphi approach

Authors : Amanda Sizo, Adriano Lino, Álvaro Rocha, Luis Paulo Reis

The quality of peer review reports is essential to the integrity and effectiveness of scholarly communication. Yet review reports are often criticized for being vague, biased, or unconstructive, which limits their usefulness for both authors and editors. Existing frameworks for assessing review quality remain fragmented and are rarely validated through expert consensus.

This study aims to define and validate a comprehensive set of quality dimensions for peer review reports, encompassing comments addressed to both authors and editors. We employed a two-phase design combining a thematic analysis of the literature with a Delphi study involving 43 scientific editors, primarily from journals in Computer Science and Engineering.

Consensus was reached after two Delphi rounds, resulting in 62 validated statements organized into eight quality dimensions: Helpfulness, Specificity, Fairness, Thoroughness, Courteousness, Readability, Consistency, and Relevance. These findings provide an empirically grounded framework to inform the development of clearer standards for peer review practice.

URL : Determining quality dimensions for peer review reports using a Delphi approach

DOI : https://doi.org/10.1007/s11192-026-05603-3

Catégories
EN

Gaming the peer review system: Evidence for a review mill in medicine highlights the need to ensure reviewer integrity

Authors : M. Ángeles Oviedo-García, René Aquarius, Dorothy V. M. Bishop

Background

Review mills are recognized when individuals generate numerous generic review reports, typically containing suggestions for citations to their own work. Here, we report a network with characteristics of a review mill in the field of gynecological oncology.

Methods

Our search started with a review that contained “boilerplate” comments as well as suggestions that specific PubMed IDs be cited. We searched the internet using Google for review reports using the same boilerplate comments. We coded text to quantify similarities between reviews and compiled citations suggested by reviewers. For comparison, we analyzed 59 reviews of the same articles by other peer reviewers.

Results

We identified a network of 195 review reports that shared boilerplate text from 170 articles. One hundred and eighty-six reports suggested citing articles coauthored by a member of the network. Five members of the network had editorial roles. Authors of 142 articles complied with suggestions for citation. Boilerplate text and citation recommendations were rare in the comparison reports.

Conclusions

Review mills lead to articles being published without proper peer review. This is of particular concern in medical research. Open peer review and transparent reporting of the editors responsible for handling papers will make it easier to detect review mills.

URL : Gaming the peer review system: Evidence for a review mill in medicine highlights the need to ensure reviewer integrity

DOI : https://doi.org/10.1080/08989621.2026.2640012

Catégories
EN

Back to the roots: reimagining scientific evaluation of research without peer review

Author : Malik Sallam

The peer review system, once a noble aspiration, now lags behind the accelerating demands of modern science.

This opinion piece calls for a decisive departure from that peer review system and advocates for a return to a more accountable, editorially driven model of scholarly evaluation. Scientific editors – already vested with decision‑making authority – should no longer outsource their judgement to external referees. Instead, they must reclaim their rightful role as the primary arbiters of scientific merit.

Too often, editorial judgement is diluted by ritualized consultation, where peer review delays innovation, rewards consensus and obscures responsibility.

I argue for a future in which academic editors decide independently, sign their decisions and are recognized – publicly and professionally – for the intellectual stewardship they provide. By linking editorial work to scientific databases such as Web of Science, Scopus and Google Scholar, we can incentivize rigour, transparency and accountability.

This model would not erode scientific integrity but elevate it, replacing bureaucracy with responsibility. It is time to shed the cloak of anonymity and return authority – and credit – to those best positioned to shape the scientific record; the academic editors themselves.

URL : Back to the roots: reimagining scientific evaluation of research without peer review

DOI : https://doi.org/10.1629/uksg.714

Catégories
EN

Peer Review at the Crossroads

Author : Dmitry Kochetkov

Peer review has long been regarded as a cornerstone of scholarly communication, ensuring high quality and credibility of published research. Although academic journals trace their origins back three centuries, the procedures for evaluating submissions, particularly peer review, have undergone continuous evolution.

Peer review’s formal institutionalisation in the mid-20th century represents a significant, yet natural, phase in this ongoing transformation of scholarly communication. By the early 21st century, there emerged an opinion that the conventional model of peer review faces systematic challenges, including inefficiency, bias and institutional inertia.

The study aims to synthesise the evolution, practices and outcomes of both conventional and innovative peer review models in scholarly publishing. Through a mixed-methods approach combining interpretative literature review and process modelling (Business Process Model and Notation–BPMN), it identifies four frameworks: pre-publication peer review, registered reports, modular publishing and the Publish-Review-Curate (PRC) model.

While the PRC model, which integrates preprints with post-publication review, demonstrates advantages in transparency and accessibility, no single approach emerges as universally ideal. The choice of model depends on disciplinary context, resource availability and institutional priorities.

The analysis underscores the need for adaptable platforms that enable hybrid workflows, balancing rigour with inclusivity. Future research must address empirical gaps in evaluating these innovations, particularly their long-term impact on equity and epistemic norms.

URL : Peer Review at the Crossroads

DOI : https://doi.org/10.1002/leap.2046

Catégories
EN

Researchers’ Views on Preprints and Open Access Publishing: Results From a Free-Answer Survey of Japanese Molecular Biologists

Authors : Harufumi Tamazawa, Kazuki Ide, Kazuhisa Kamegai

A survey conducted in 2022 amongst members of the Molecular Biology Society of Japan (n = 633) about preprints and open access journals included qualitative data from free-response answers (n = 161). Analysis of the free-form responses suggests that researchers believe that peer review of papers is the foundation for ensuring the credibility of research content.

The trust-building mechanism achieved through peer review shapes the research community. For this reason, researchers are extremely cautious about preprints that have not undergone peer review within their own fields.

This foundation has fostered a sense of responsibility within the community, and this sense of responsibility, which is being fulfilled by ensuring the quality of research, is a mixture of both a sense of responsibility towards the community itself and a sense of responsibility towards the outside world, namely the relationship between researchers and society.

Researchers also appear to view the rise in Article Processing Charges (APCs) as a problem for the entire community, rather than simply an issue for individual researchers. In the field of molecular biology, where collaborative research between universities and companies is common, differences in normative awareness based on position are reflected in the various attitudes towards preprints and open access.

URL : Researchers’ Views on Preprints and Open Access Publishing: Results From a Free-Answer Survey of Japanese Molecular Biologists

DOI : https://doi.org/10.1002/leap.2039