The experiences of COVID-19 preprint authors: a survey of researchers about publishing and receiving feedback on their work during the pandemic

Authors : Narmin Rzayeva, Susana Oliveira Henriques, Stephen Pinfield, Ludo Waltman

The COVID-19 pandemic caused a rise in preprinting, triggered by the need for open and rapid dissemination of research outputs. We surveyed authors of COVID-19 preprints to learn about their experiences with preprinting their work and also with publishing their work in a peer-reviewed journal.

Our research had the following objectives: 1. to learn about authors’ experiences with preprinting, their motivations, and future intentions; 2. to consider preprints in terms of their effectiveness in enabling authors to receive feedback on their work; 3. to compare the impact of feedback on preprints with the impact of comments of editors and reviewers on papers submitted to journals. In our survey, 78% of the new adopters of preprinting reported the intention to also preprint their future work.

The boost in preprinting may therefore have a structural effect that will last after the pandemic, although future developments will also depend on other factors, including the broader growth in the adoption of open science practices. A total of 53% of the respondents reported that they had received feedback on their preprints. However, more than half of the feedback was received through “closed” channels–privately to the authors.

This means that preprinting was a useful way to receive feedback on research, but the value of feedback could be increased further by facilitating and promoting “open” channels for preprint feedback. Almost a quarter of the feedback received by respondents consisted of detailed comments, showing the potential of preprint feedback to provide valuable comments on research.

Respondents also reported that, compared to preprint feedback, journal peer review was more likely to lead to major changes to their work, suggesting that journal peer review provides significant added value compared to feedback received on preprints.

URL : The experiences of COVID-19 preprint authors: a survey of researchers about publishing and receiving feedback on their work during the pandemic

DOI : https://doi.org/10.7717/peerj.15864

Comparison of effect estimates between preprints and peer-reviewed journal articles of COVID-19 trials

Authors : Mauricia Davidson, Theodoros Evrenoglou, Carolina Graña, Anna Chaimani, Isabelle Boutron

Background

Preprints are increasingly used to disseminate research results, providing multiple sources of information for the same study. We assessed the consistency in effect estimates between preprint and subsequent journal article of COVID-19 randomized controlled trials.

Methods

The study utilized data from the COVID-NMA living systematic review of pharmacological treatments for COVID-19 (covid-nma.com) up to July 20, 2022. We identified randomized controlled trials (RCTs) evaluating pharmacological treatments vs. standard of care/placebo for patients with COVID-19 that were originally posted as preprints and subsequently published as journal articles.

Trials that did not report the same analysis in both documents were excluded. Data were extracted independently by pairs of researchers with consensus to resolve disagreements. Effect estimates extracted from the first preprint were compared to effect estimates from the journal article.

Results

The search identified 135 RCTs originally posted as a preprint and subsequently published as a journal article. We excluded 26 RCTs that did not meet the eligibility criteria, of which 13 RCTs reported an interim analysis in the preprint and a final analysis in the journal article. Overall, 109 preprint–article RCTs were included in the analysis.

The median (interquartile range) delay between preprint and journal article was 121 (73–187) days, the median sample size was 150 (71–464) participants, 76% of RCTs had been prospectively registered, 60% received industry or mixed funding, 72% were multicentric trials. The overall risk of bias was rated as ‘some concern’ for 80% of RCTs.

We found that 81 preprint–article pairs of RCTs were consistent for all outcomes reported. There were nine RCTs with at least one outcome with a discrepancy in the number of participants with outcome events or the number of participants analyzed, which yielded a minor change in the estimate of the effect. Furthermore, six RCTs had at least one outcome missing in the journal article and 14 RCTs had at least one outcome added in the journal article compared to the preprint. There was a change in the direction of effect in one RCT. No changes in statistical significance or conclusions were found.

Conclusions

Effect estimates were generally consistent between COVID-19 preprints and subsequent journal articles. The main results and interpretation did not change in any trial. Nevertheless, some outcomes were added and deleted in some journal articles.

URL : Comparison of effect estimates between preprints and peer-reviewed journal articles of COVID-19 trials

DOI : https://doi.org/10.1186/s12874-023-02136-8

The impact of COVID-19 on the debate on open science: An analysis of expert opinion

Auteurs/Authors : Melanie Benson Marshall,  Stephen Pinfield, Pamela Abbott, Andrew Cox, Juan Pablo Alperin,  Natascha Chtena, Isabelle Dorsch, Alice Fleerackers, Monique Oliveira,
Isabella Peters

This study is an analysis of the international debate on open science that took place during the pandemic. It addresses the question, how did the COVID-19 pandemic impact the debate on open science?

The study takes the form of a qualitative analysis of a large corpus of key articles, editorials, blogs and thought pieces about the impact of COVID on open science, published during the pandemic in English, German, Portuguese, and Spanish.

The findings show that many authors believed that it was clear that the experience of the pandemic had illustrated or strengthened the case for open science, with language such as a “stress test”, “catalyst”, “revolution” or “tipping point” frequently used. It was commonly believed that open science had played a positive role in the response to the pandemic, creating a clear ‘line of sight’ between open science and societal benefits.

Whilst the arguments about open science deployed in the debate were not substantially new, the focuses of debate changed in some key respects. There was much less attention given to business models for open access and critical perspectives on open science, but open data sharing, preprinting, information quality and misinformation became most prominent in debates. There were also moves to reframe open science conceptually, particularly in connecting science with society and addressing broader questions of equity.

The impact of COVID-19 on the debate on open science: An analysis of expert opinion

DOI : https://doi.org/10.31235/osf.io/xy874

Making science public: a review of journalists’ use of Open Science research

Authors : Alice Fleerackers, Natascha Chten, Stephen Pinfield, Juan Pablo Alperin, Germana Barata, Monique Oliveira, Isabella Peters

Science journalists are uniquely positioned to increase the societal impact of open science by contextualizing and communicating research findings in ways that highlight their relevance and implications for non-specialist audiences.

Through engagement with and coverage of open research outputs, journalists can help align the ideals of openness, transparency, and accountability with the wider public sphere and its democratic potential.

Yet, it is unclear to what degree journalists use open research outputs in their reporting, what factors motivate or constrain this use, and how the recent surge in openly available research seen during the COVID-19 pandemic has affected the relationship between open science and science journalism.

This literature review thus examines journalists’ use of open research outputs, specifically open access publications and preprints. We focus on literature published from 2018 onwards—particularly literature relating to the COVID-19 pandemic—but also include seminal articles outside the search dates.

We find that, despite journalists’ potential to act as critical brokers of open access knowledge, their use of open research outputs is hampered by an overreliance on traditional criteria for evaluating scientific quality; concerns about the trustworthiness of open research outputs; and challenges using and verifying the findings.

We also find that, while the COVID-19 pandemic encouraged journalists to explore open research outputs such as preprints, the extent to which these explorations will become established journalistic practices remains unclear.

Furthermore, we note that current research is overwhelmingly authored and focused on the Global North, and the United States specifically.

Finally, given the dearth of research in this area, we conclude with recommendations for future research that attend to issues of equity and diversity, and more explicitly examine the intersections of open science and science journalism.

URL : Making science public: a review of journalists’ use of Open Science research

DOI : https://doi.org/10.12688/f1000research.133710.1

The pandemic and changes in early career researchers’ career prospects, research and publishing practices

Authors : Hamid R. Jamali, David Nicholas, David Sims, Anthony Watkinson, Eti Herman, Cherifa Boukacem-Zeghmouri, Blanca Rodrıguez-Bravo, Marzena Świgoń, Abdullah Abrizah, Jie Xu, Carol Tenopir , Suzie Allard

Introduction

As part of the Harbnger-2 project, this study aimed to discover the impact of the COVID-19 pandemic on junior researchers’ work-life, career prospects, research and publishing practices and networking.

Methods

An online international survey of 800 early career researchers (ECRs) was conducted in 2022. A questionnaire was developed based on three rounds of interviews and distributed using multiple channels including publishers, social media, and direct email to ECRs.

Results

The impact of the pandemic on career prospects, morale, job security, productivity, ability to network and collaborate, and quality and speed of peer review has on the whole been more negative than positive.

A quarter of ECRs shifted their research focus to pandemic-related topics and half of those who did, benefited largely due to increased productivity and impact. The majority worked remotely/from home and more than two-thirds of those who did so benefitted from it. While virtual or hybrid conferences have been embraced by the majority of ECRs, around a third still preferred face-to-face only conferences.

The use of library online platforms, Sci-Hub, ResearchGate, Google Scholar and smartphone to search and access full-text papers increased. ECRs prioritised journals with fast submission procedures for the publishing of their papers and spent more time on increasing the visibility of their research. Fees were a problem for publishing open access.

Conclusion

Although, generally, the pandemic negatively impacted many aspects of ECRs’ work-life, certain research areas and individuals benefited from being more appreciated and valued, and, in some cases, resulted in increased resources, better productivity and greater impact.

Changes, such as the use of digital technologies and remote working created new opportunities for some ECRs. While continuing work flexibility and hybrid conferences might benefit some ECRs, institutions should also take measures to help those ECRs whose career and productivity have been adversely impacted.

URL : The pandemic and changes in early career researchers’ career prospects, research and publishing practices

DOI : https://doi.org/10.1371/journal.pone.0281058

Interdisciplinary Analysis of Science Communication on Social Media during the COVID-19 Crisis

Authors : Thomas Mandl, Sylvia Jaki, Hannah Mitera, Franziska Schmidt

In times of crisis, science communication needs to be accessible and convincing. In order to understand whether these two criteria apply to concrete science communication formats, it is not enough to merely study the communication product. Instead, the recipient’s perspective also needs to be taken into account.

What do recipients value in popular science communication formats concerning COVID-19? What do they criticize? What elements in the formats do they pay attention to? These questions can be answered by reception studies, for example, by analyzing the reactions and comments of social media users.

This is particularly relevant since scientific information was increasingly disseminated over social media channels during the COVID-19 crisis. This interdisciplinary study, therefore, focuses both on science communication strategies in media formats and the related comments on social media.

First, we selected science communication channels on YouTube and performed a qualitative multi-modal analysis. Second, the comments responding to science communication content online were analyzed by identifying Twitter users who are doctors, researchers, science communicators and those who represent research institutes and then, subsequently, performing topic modeling on the textual data.

The main goal was to find topics that directly related to science communication strategies. The qualitative video analysis revealed, for example, a range of strategies for accessible communication and maintaining transparency about scientific insecurities.

The quantitative Twitter analysis showed that few tweets commented on aspects of the communication strategies. These were mainly positive while the sentiment in the overall collection was less positive.

We downloaded and processed replies for 20 months, starting at the beginning of the pandemic, which resulted in a collection of approximately one million tweets from the German science communication market.

URL : Interdisciplinary Analysis of Science Communication on Social Media during the COVID-19 Crisis

DOI : https://doi.org/10.3390/knowledge3010008

Model(s) of the future? Overlay journals as an overlooked and emerging trend in scholarly communication

Authors : Gail M. Thornton, Emily Kroeker

Overlay journals, a potentially overlooked model of scholarly communication, have seen a resurgence due to the increasing number of preprint repositories and preprints on coronavirus disease 2019 (COVID-19) related topics.

Overlay journals at various stages of maturity were examined for unique characteristics, including whether the authors submitted their article to the journal, whether the peer reviews of the article were published by the overlay journal, and whether the overlay journals took advantage of opportunities for increased discovery.

As librarians and researchers seek new, futuristic models for publishing, overlay journals are emerging as an important contribution to scholarly communication.

URL : Model(s) of the future? Overlay journals as an overlooked and emerging trend in scholarly communication

DOI : https://doi.org/10.5206/cjils-rcsib.v45i2.14730