Étiquette : Peer Review

Catégories
EN

Peer Review at the Crossroads

Author : Dmitry Kochetkov

Peer review has long been regarded as a cornerstone of scholarly communication, ensuring high quality and credibility of published research. Although academic journals trace their origins back three centuries, the procedures for evaluating submissions, particularly peer review, have undergone continuous evolution.

Peer review’s formal institutionalisation in the mid-20th century represents a significant, yet natural, phase in this ongoing transformation of scholarly communication. By the early 21st century, there emerged an opinion that the conventional model of peer review faces systematic challenges, including inefficiency, bias and institutional inertia.

The study aims to synthesise the evolution, practices and outcomes of both conventional and innovative peer review models in scholarly publishing. Through a mixed-methods approach combining interpretative literature review and process modelling (Business Process Model and Notation–BPMN), it identifies four frameworks: pre-publication peer review, registered reports, modular publishing and the Publish-Review-Curate (PRC) model.

While the PRC model, which integrates preprints with post-publication review, demonstrates advantages in transparency and accessibility, no single approach emerges as universally ideal. The choice of model depends on disciplinary context, resource availability and institutional priorities.

The analysis underscores the need for adaptable platforms that enable hybrid workflows, balancing rigour with inclusivity. Future research must address empirical gaps in evaluating these innovations, particularly their long-term impact on equity and epistemic norms.

URL : Peer Review at the Crossroads

DOI : https://doi.org/10.1002/leap.2046

Catégories
EN

Researchers’ Views on Preprints and Open Access Publishing: Results From a Free-Answer Survey of Japanese Molecular Biologists

Authors : Harufumi Tamazawa, Kazuki Ide, Kazuhisa Kamegai

A survey conducted in 2022 amongst members of the Molecular Biology Society of Japan (n = 633) about preprints and open access journals included qualitative data from free-response answers (n = 161). Analysis of the free-form responses suggests that researchers believe that peer review of papers is the foundation for ensuring the credibility of research content.

The trust-building mechanism achieved through peer review shapes the research community. For this reason, researchers are extremely cautious about preprints that have not undergone peer review within their own fields.

This foundation has fostered a sense of responsibility within the community, and this sense of responsibility, which is being fulfilled by ensuring the quality of research, is a mixture of both a sense of responsibility towards the community itself and a sense of responsibility towards the outside world, namely the relationship between researchers and society.

Researchers also appear to view the rise in Article Processing Charges (APCs) as a problem for the entire community, rather than simply an issue for individual researchers. In the field of molecular biology, where collaborative research between universities and companies is common, differences in normative awareness based on position are reflected in the various attitudes towards preprints and open access.

URL : Researchers’ Views on Preprints and Open Access Publishing: Results From a Free-Answer Survey of Japanese Molecular Biologists

DOI : https://doi.org/10.1002/leap.2039

Catégories
EN

Opening Pandora’s box: Developing reviewer rhetorical sensitivity through retracted articles

Author : Baraa Khuder

Retractions issued for misconduct offer a unique window into how questionable research is rhetorically constructed and made to appear credible. This study investigates how engaging with retracted articles can serve as a pedagogical tool for reviewer training, with particular attention to the rhetorical mechanisms through which unreliability is performed.

Twenty STEM doctoral researchers analyzed self-selected retracted papers using guided critical-reading questions to identify problematic rhetorical features. Across the analyses, five recurring issues emerged: intertextual falsification, methodological opacity, rhetorical inconsistency, rhetorical overstatement, and terminological distortion.

The findings indicate that this approach has the potential to raise doctoral students’ rhetorical sensitivity by enabling them to detect subtle markers of unreliability and to adopt a more evaluative rhetorical stance toward scholarly texts.

Retracted articles thus can provide an authentic pedagogical resource for developing reviewer rhetorical sensitivity within doctoral education.

URL : Opening Pandora’s box: Developing reviewer rhetorical sensitivity through retracted articles

DOI : https://doi.org/10.1080/08989621.2025.2607681

 

Catégories
EN

A Cross-Disciplinary Analysis of AI Policies in Academic Peer Review

Authors : Zhongshi Wang, Mengyue Gong

Rapid advances of artificial intelligence (AI) have substantially impacted the field of academic publishing. This study examines AI integration in peer review by analysing policies from 439 high- and 363 middle-impact factor (IF) journals across disciplines. Using grounded theory, we identify patterns in AI policy adoption.

Results show 83% of high-IF journals have AI guidelines, with varying stringency across disciplines. Meanwhile, only 75% of middle-IF journals have AI guidelines. Science, technology, and medicine (STM) disciplines exhibit stricter regulations, while humanities and social sciences adopt more lenient approaches.

Key ethical concerns focus on confidentiality risks, accountability gaps, and AI’s inability to replicate critical human judgement. Publisher policies emphasise transparency, human oversight, and restricted AI usage for auxiliary tasks only, such as grammar checks or reviewer finding.

Disciplinary differences highlight the need for tailored guidelines that balance efficiency gains with research integrity. This study proposes collaborative frameworks for responsible AI integration. It focuses on accountability, transparency, and interdisciplinary policy development to address peer review challenges.

URL : A Cross-Disciplinary Analysis of AI Policies in Academic Peer Review

DOI : https://doi.org/10.1002/leap.2035

Catégories
EN

Enhancing Consistency in Peer Review: A Statistical Analysis of Discrepancies and Proposals for Improvement

Authors : Maher M. Alarfaj

This paper investigates the inconsistencies present in peer review by analysing the evaluation patterns of reviewers involved in an educational award in the Arab Gulf Country States. A statistical approach was used to assess the degree of variation in scores assigned to 270 manuscripts reviewed by three different groups of reviewers.

The study revealed significant differences in the evaluations, suggesting that at least two reviewers often showed discrepancies in their assessments despite using the standardised evaluation form. The observed discrepancies appear to reflect underlying complexities related to reviewer perspectives and evaluation standards, highlighting challenges in achieving uniformity across assessments.

Additionally, it proposes a model to enhance peer review consistency, including methods for score adjustment and calibration to mitigate reviewer differences. The goal is to offer practical recommendations for improving the fairness, transparency and reliability of peer review systems, contributing to the ongoing development of academic publishing practices.

For audiences beyond the academic community including publishers, editors and academic librarians, these findings show how practical statistical tools can strengthen peer review and build greater trust in academic publishing.

URL : Enhancing Consistency in Peer Review: A Statistical Analysis of Discrepancies and Proposals for Improvement

DOI : https://doi.org/10.1002%2Fleap.2033

Catégories
EN

Reforming research funding: Combining editorial preregistration with grant peer review

Authors : Lutz Bornmann, Gerald Schweiger

Competitive grant funding is associated with high costs and a potential bias to favor conservative research. This comment proposes integrating editorial preregistration, in the form of registered reports, into grant peer review processes as a reform strategy.

Linking funding decisions to in principle accepted study protocols would reduce reviewer burden, strengthen methodological rigor, and provide an institutional foundation for (more) replication, theory driven research, and high risk research. Our proposal also minimizes strategic proposal writing and ensures scholarly output through the publication of preregistered protocols, regardless of funding outcomes.

Possible implementation models include direct coupling of journal acceptance with funding, co review mechanisms, voucher systems, and lotteries. While challenges remain in aligning journal and funding agency procedures, the integration of preregistration and funding offers a promising pathway toward a more transparent and efficient research ecosystem.

DOI : https://doi.org/10.48550/arXiv.2511.01439

Catégories
EN

Scientific publishing without gatekeeping: an empirical investigation of eLife’s new peer review process

Authors : Rüdiger Mutz, Lutz Bornmann, Hans‑Dieter Daniel

At the end of January 2023, eLife introduced a new publishing model (alongside the old-traditional-publishing model): all manuscripts submitted as preprints are peer-reviewed and published if they are deemed worthy of review by the editorial team (“editorial triage”). The model abandons the gatekeeping function and retains the previous “consultative approach to peer review”.

Even under the changed conditions, the question of the quality of judgements in the peer review process remains. In this study, the reviewers’ ratings of manuscripts submitted to eLife were examined in terms of both descriptive comparisons of peer review models, and the following selected quality criteria of peer review: interrater agreement and interrater reliability. eLife provided us with the data on all manuscripts submitted in 2023 according to the new publishing model (group 3, N = 3,846), as well as manuscripts submitted according to the old publishing model (group 1: N = 6,592 submissions from 2019; group 2: N = 364 submissions from 2023).

The interrater agreement and interrater reliability for the criteria “significance of findings” and “strength of support” were similarly low, as previous empirical studies for gatekeeping journals have shown.

The fairness of peer review is not or only slightly compromised. We used the empirical results of our study to recommend several improvements to the new publishing model introduced by eLife as for example, increasing transparency, masking author identity or increasing the number of expert reviewers.

URL : Scientific publishing without gatekeeping: an empirical investigation of eLife’s new peer review process

DOI : https://doi.org/10.1007/s11192-025-05422-y