Catégories
EN

Gaming the peer review system: Evidence for a review mill in medicine highlights the need to ensure reviewer integrity

Authors : M. Ángeles Oviedo-García, René Aquarius, Dorothy V. M. Bishop

Background

Review mills are recognized when individuals generate numerous generic review reports, typically containing suggestions for citations to their own work. Here, we report a network with characteristics of a review mill in the field of gynecological oncology.

Methods

Our search started with a review that contained “boilerplate” comments as well as suggestions that specific PubMed IDs be cited. We searched the internet using Google for review reports using the same boilerplate comments. We coded text to quantify similarities between reviews and compiled citations suggested by reviewers. For comparison, we analyzed 59 reviews of the same articles by other peer reviewers.

Results

We identified a network of 195 review reports that shared boilerplate text from 170 articles. One hundred and eighty-six reports suggested citing articles coauthored by a member of the network. Five members of the network had editorial roles. Authors of 142 articles complied with suggestions for citation. Boilerplate text and citation recommendations were rare in the comparison reports.

Conclusions

Review mills lead to articles being published without proper peer review. This is of particular concern in medical research. Open peer review and transparent reporting of the editors responsible for handling papers will make it easier to detect review mills.

URL : Gaming the peer review system: Evidence for a review mill in medicine highlights the need to ensure reviewer integrity

DOI : https://doi.org/10.1080/08989621.2026.2640012

Catégories
EN

‘Nepotistic journals’: a survey of biomedical journals

Authors : Alexandre Scanff, Florian Naudet, Ioana Cristea, David Moher, Dorothy V M Bishop, Clara Locher

Context

Convergent analyses in different disciplines support the use of the Percentage of Papers by the Most Prolific author (PPMP) as a red flag to identify journals that can be suspected of questionable editorial practices. We examined whether this index, complemented by the Gini index, could be useful for identifying cases of potential editorial bias, using a large sample of biomedical journals.

Methods

We extracted metadata for all biomedical journals referenced in the National Library of Medicine, with any attributed Broad Subject Terms, and at least 50 authored (i.e. by at least one author) articles between 2015 and 2019, identifying the most prolific author (i.e. the person who signed the most papers in each particular journal).

We calculated the PPMP and the 2015-2019 Gini index for the distribution of articles across authors. When the relevant information was reported, we also computed the median publication lag (time between submission and acceptance) for articles authored by any of the most prolific authors and that for articles not authored by prolific authors.

For outlier journals, defined as a PPMP or Gini index above the 95th percentile of their respective distributions, a random sample of 100 journals was selected and described in relation to status on the editorial board for the most prolific author.

Results

5 468 journals that published 4 986 335 papers between 2015 and 2019 were analysed. The PPMP 95th percentile was 10.6% (median 2.9%). The Gini index 95th percentile was 0.355 (median 0.183). Correlation between the two indices was 0.35 (95CI 0.33 to 0.37). Information on publication lag was available for 2 743 journals.

We found that 277 journals (10.2%) had a median time lag to publication for articles by the most prolific author(s) that was shorter than 3 weeks, versus 51 (1.9%) journals with articles not authored by prolific author(s).

Among the random sample of outlier journals, 98 provided information about their editorial board. Among these 98, the most prolific author was part of the editorial board in 60 cases (61%), among whom 25 (26% of the 98) were editors-in-chief.

Discussion

In most journals publications are distributed across a large number of authors. Our results reveal a subset of journals where a few authors, often members of the editorial board, were responsible for a disproportionate number of publications.

The papers by these authors were more likely to be accepted for publication within 3 weeks of their submission. To enhance trust in their practices, journals need to be transparent about their editorial and peer review practices.

URL : ‘Nepotistic journals’: a survey of biomedical journals

DOI : https://doi.org/10.1101/2021.02.03.429520