Examining linguistic shifts between preprints and publications

Authors : David N. Nicholson, Vincent Rubinetti, Dongbo Hu, Marvin Thielk, Lawrence E. Hunter, Casey S. Greene

Preprints allow researchers to make their findings available to the scientific community before they have undergone peer review. Studies on preprints within bioRxiv have been largely focused on article metadata and how often these preprints are downloaded, cited, published, and discussed online.

A missing element that has yet to be examined is the language contained within the bioRxiv preprint repository. We sought to compare and contrast linguistic features within bioRxiv preprints to published biomedical text as a whole as this is an excellent opportunity to examine how peer review changes these documents.

The most prevalent features that changed appear to be associated with typesetting and mentions of supporting information sections or additional files. In addition to text comparison, we created document embeddings derived from a preprint-trained word2vec model.

We found that these embeddings are able to parse out different scientific approaches and concepts, link unannotated preprint–peer-reviewed article pairs, and identify journals that publish linguistically similar papers to a given preprint.

We also used these embeddings to examine factors associated with the time elapsed between the posting of a first preprint and the appearance of a peer-reviewed publication. We found that preprints with more versions posted and more textual changes took longer to publish.

Lastly, we constructed a web application (https://greenelab.github.io/preprint-similarity-search/) that allows users to identify which journals and articles that are most linguistically similar to a bioRxiv or medRxiv preprint as well as observe where the preprint would be positioned within a published article landscape.

URL : Examining linguistic shifts between preprints and publications

DOI : https://doi.org/10.1371/journal.pbio.3001470

Publishing of COVID-19 preprints in peer-reviewed journals, preprinting trends, public discussion and quality issues

Authors : Ivan Kodvanj, Jan Homolak, Vladimir Trkulja

COVID-19-related (vs. non-related) articles appear to be more expeditiously processed and published in peer-reviewed journals.

We aimed to evaluate: (i) whether COVID-19-related preprints were favored for publication, (ii) preprinting trends and public discussion of the preprints, and (iii) the relationship between the publication topic (COVID-19-related or not) and quality issues.

Manuscripts deposited at bioRxiv and medRxiv between January 1 and September 27 2020 were assessed for the probability of publishing in peer-reviewed journals, and those published were evaluated for submission-to-acceptance time. The extent of public discussion was assessed based on Altmetric and Disqus data.

The Retraction Watch Database and PubMed were used to explore the retraction of COVID-19 and non-COVID-19 articles and preprints. With adjustment for the preprinting server and number of deposited versions, COVID-19-related preprints were more likely to be published within 120 days since the deposition of the first version (OR = 1.96, 95% CI: 1.80–2.14) as well as over the entire observed period (OR = 1.39, 95% CI: 1.31–1.48). Submission-to-acceptance was by 35.85 days (95% CI: 32.25–39.45) shorter for COVID-19 articles.

Public discussion of preprints was modest and COVID-19 articles were overrepresented in the pool of retracted articles in 2020. Current data suggest a preference for publication of COVID-19-related preprints over the observed period.

URL : https://doi.org/10.1007/s11192-021-04249-7

Characteristics of retracted editorial articles in the biomedical literature

Author : Bakthavachalam Elango

The main purpose of this short communication is to identify and analyze retracted editorials in the biomedical literature. Twenty-five of the 33 editorials are chosen for further analysis. All of the editorials were published as commentaries and concise reviews between 1998 and 2021.

Due to plagiarism and data-related issues, the majority of the editorial articles were retracted. Alarmingly, one-fifth of the editorials were funded by external agencies, with the USA leading the list of retracted editorials.

The average time between the publication of the editorials and their retraction is 2.48 years, and two editorials were retracted with the longest durations; both were written by the same author.

The conclusion is that, at the very least, editorial articles should be devoid of research/scientific misconduct.

DOI : https://doi.org/10.1007/s11192-021-04263-9

Impact of the COVID-19 pandemic on publication dynamics and non-COVID-19 research production

Authors : Marc Raynaud, Valentin Goutaudier, Kevin Louis, Solaf Al‑Awadhi, Quentin Dubourg, Agathe Truchot, Romain Brousse, Nouredine Saleh, Alessia Giarraputo, Charlotte Debiais, Zeynep Demir, Anaïs Certain, Francine Tacafred, Esteban Cortes‑Garcia, Safia Yanes, Jessy Dagobert, Sofia Naser, Blaise Robin, Élodie Bailly, Xavier Jouven, Peter P. Reese, Alexandre Loupy

Background

The COVID-19 pandemic has severely affected health systems and medical research worldwide but its impact on the global publication dynamics and non-COVID-19 research has not been measured.

We hypothesized that the COVID-19 pandemic may have impacted the scientific production of non-COVID-19 research.

Methods

We conducted a comprehensive meta-research on studies (original articles, research letters and case reports) published between 01/01/2019 and 01/01/2021 in 10 high-impact medical and infectious disease journals (New England Journal of Medicine, Lancet, Journal of the American Medical Association, Nature Medicine, British Medical Journal, Annals of Internal Medicine, Lancet Global Health, Lancet Public Health, Lancet Infectious Disease and Clinical Infectious Disease).

For each publication, we recorded publication date, publication type, number of authors, whether the publication was related to COVID-19, whether the publication was based on a case series, and the number of patients included in the study if the publication was based on a case report or a case series.

We estimated the publication dynamics with a locally estimated scatterplot smoothing method. A Natural Language Processing algorithm was designed to calculate the number of authors for each publication.

We simulated the number of non-COVID-19 studies that could have been published during the pandemic by extrapolating the publication dynamics of 2019 to 2020, and comparing the expected number to the observed number of studies.

Results

Among the 22,525 studies assessed, 6319 met the inclusion criteria, of which 1022 (16.2%) were related to COVID-19 research. A dramatic increase in the number of publications in general journals was observed from February to April 2020 from a weekly median number of publications of 4.0 (IQR: 2.8–5.5) to 19.5 (IQR: 15.8–24.8) (p < 0.001), followed afterwards by a pattern of stability with a weekly median number of publications of 10.0 (IQR: 6.0–14.0) until December 2020 (p = 0.045 in comparison with April).

Two prototypical editorial strategies were found: 1) journals that maintained the volume of non-COVID-19 publications while integrating COVID-19 research and thus increased their overall scientific production, and 2) journals that decreased the volume of non-COVID-19 publications while integrating COVID-19 publications.

We estimated using simulation models that the COVID pandemic was associated with a 18% decrease in the production of non-COVID-19 research. We also found a significant change of the publication type in COVID-19 research as compared with non-COVID-19 research illustrated by a decrease in the number of original articles, (47.9% in COVID-19 publications vs 71.3% in non-COVID-19 publications, p < 0.001).

Last, COVID-19 publications showed a higher number of authors, especially for case reports with a median of 9.0 authors (IQR: 6.0–13.0) in COVID-19 publications, compared to a median of 4.0 authors (IQR: 3.0–6.0) in non-COVID-19 publications (p < 0.001).

Conclusion

In this meta-research gathering publications from high-impact medical journals, we have shown that the dramatic rise in COVID-19 publications was accompanied by a substantial decrease of non-COVID-19 research.

URL : Impact of the COVID-19 pandemic on publication dynamics and non-COVID-19 research production

DOI : https://doi.org/10.1186/s12874-021-01404-9

How far does an emphasis on stakeholder engagement and co-production in research present a threat to academic identity and autonomy? A prospective study across five European countries

Authors : Annette Boaz, Robert Borst, Maarten Kok, Alison O’Shea

There is a growing recognition that needs more to be done to ensure that research contributes to better health services and patient outcomes. Stakeholder engagement in research, including co-production, has been identified as a promising mechanism for improving the value, relevance and utilization of research.

This article presents findings from a prospective study which explored the impact of stakeholder engagement in a 3-year European tobacco control research project. That research project aimed to engage stakeholders in the development, testing and dissemination of a return-on-investment tool across five EU countries (the Netherlands, Spain, Hungary, Germany and the UK).

The prospective study comprised interviews, observations and document review. The analysis focused on the extent to which the project team recognized, conceptualized and operationalized stakeholder engagement over the course of the research project. Stakeholder engagement in the European research project was conceptualized as a key feature of pre-designated spaces within their work programme.

Over the course of the project, however, the tool development work and stakeholder engagement activities decoupled. While the modelling and tool development became more secluded, stakeholder engagement activities subtly transformed from co-production, to consultation, to something more recognizable as research participation.

The contribution of this article is not to argue against the potential contribution of stakeholder engagement and co-production, but to show how even well-planned engagement activities can be diverted within the existing research funding and research production systems where non-research stakeholders remain at the margins and can even be seen as a threat to academic identify and autonomy.

URL : How far does an emphasis on stakeholder engagement and co-production in research present a threat to academic identity and autonomy? A prospective study across five European countries

DOI : https://doi.org/10.1093/reseval/rvab013

Investigating the process of ethical approval in citizen science research: the case of Public Health

Authors : Antonella Ficorilli, Giovanni Maccani, Mara Balestrini, Annibale Biggeri, Bruna De Marchi, Frederique E. M. Froeling, Florence Gignac, Regina Grazuleviciene, Gerard Hoek, Tjaša Kanduč, David Kocman, Valeria Righi, Xavier Basagana

Undertaking citizen science research in Public Health involving human subjects poses significant challenges concerning the traditional process of ethical approval.

It requires an extension of the ethics of protection of research subjects in order to include the empowerment of citizens as citizen scientists.

This paper investigates these challenges and illustrates the ethical framework and the strategies developed within the CitieS-Health project. It also proposes first recommendations generated from the experiences of five citizen science pilot studies in environmental epidemiology within this project.

URL : Investigating the process of ethical approval in citizen science research: the case of Public Health

DOI : https://doi.org/10.22323/2.20060204

Representation of Women Among Editors in Chief of Leading Medical Journals

Authors : Ana-Catarina Pinho-Gomes, Amy Vassallo, Kelly Thompson, Kate Womersley, Robyn Norton, Mark Woodward

Importance

Women remain underrepresented among editors of scientific journals, particularly in senior positions. However, to what extent this applies to medical journals of different specialties remains unclear.

Objective

To investigate the gender distribution of the editors in chief at leading medical journals.

Design, Setting, and Participants

Cross-sectional study of the editors in chief at the top 10 international medical journals of 41 categories related to the medical specialties of the Clarivate Analytics Web of Science Journal Citation Reports in 2019.

Main Outcomes and Measures

Proportion of women as editors in chief.

Results

This study found that, overall, women represented 21% (94 of 44) of the editors in chief, with wide variation across medical specialties from 0% to 82%. There were 5 categories for which none of the editors in chief were women (dentistry, oral surgery and medicine; allergy; psychiatry; anesthesiology; and ophthalmology) and only 3 categories for which women outnumbered men as editors in chief (primary health care, microbiology, and genetics and heredity).

In 27 of the 41 categories, women represented less than a third of the editors in chief (eg, 1 of 10 for critical care medicine, 2 of 10 for gastroenterology and hepatology, and 3 of 10 for endocrinology and metabolism).

Conclusions and Relevance

This study found that women are underrepresented among editors in chief of leading medical journals. For the benefit of medical research, a joint effort from editorial boards, publishers, authors, and academic institutions is required to address this gender gap.

URL : Representation of Women Among Editors in Chief of Leading Medical Journals

DOI :10.1001/jamanetworkopen.2021.23026