Communicating Scientific Uncertainty in an Age of COVID-19: An Investigation into the Use of Preprints by Digital Media Outlets

Authors : Alice Fleerackers, Michelle Riedlinger, Laura Moorhead, Rukhsana Ahmed, Juan Pablo Alperin

In this article, we investigate the surge in use of COVID-19-related preprints by media outlets. Journalists are a main source of reliable public health information during crises and, until recently, journalists have been reluctant to cover preprints because of the associated scientific uncertainty.

Yet, uploads of COVID-19 preprints and their uptake by online media have outstripped that of preprints about any other topic. Using an innovative approach combining altmetrics methods with content analysis, we identified a diversity of outlets covering COVID-19-related preprints during the early months of the pandemic, including specialist medical news outlets, traditional news media outlets, and aggregators.

We found a ubiquity of hyperlinks as citations and a multiplicity of framing devices for highlighting the scientific uncertainty associated with COVID-19 preprints. These devices were rarely used consistently (e.g., mentioning that the study was a preprint, unreviewed, preliminary, and/or in need of verification).

About half of the stories we analyzed contained framing devices emphasizing uncertainty. Outlets in our sample were much less likely to identify the research they mentioned as preprint research, compared to identifying it as simply “research.” This work has significant implications for public health communication within the changing media landscape.

While current best practices in public health risk communication promote identifying and promoting trustworthy sources of information, the uptake of preprint research by online media presents new challenges.

At the same time, it provides new opportunities for fostering greater awareness of the scientific uncertainty associated with health research findings.

DOI : https://doi.org/10.1080/10410236.2020.1864892

Citation needed? Wikipedia and the COVID-19 pandemic

Authors : Omer Benjakob, Rona Aviram, Jonathan Sobel

With the COVID-19 pandemic’s outbreak at the beginning of 2020, millions across the world flocked to Wikipedia to read about the virus. Our study offers an in-depth analysis of the scientific backbone supporting Wikipedia’s COVID-19 articles.

Using references as a readout, we asked which sources informed Wikipedia’s growing pool of COVID-19-related articles during the pandemic’s first wave (January-May 2020). We found that coronavirus-related articles referenced trusted media sources and cited high-quality academic research.

Moreover, despite a surge in preprints, Wikipedia’s COVID-19 articles had a clear preference for open-access studies published in respected journals and made little use of non-peer-reviewed research up-loaded independently to academic servers.

Building a timeline of COVID-19 articles on Wikipedia from 2001-2020 revealed a nuanced trade-off between quality and timeliness, with a growth in COVID-19 article creation and citations, from both academic research and popular media.

It further revealed how preexisting articles on key topics related to the virus created a frame-work on Wikipedia for integrating new knowledge. This “scientific infrastructure” helped provide context, and regulated the influx of new information into Wikipedia.

Lastly, we constructed a network of DOI-Wikipedia articles, which showed the landscape of pandemic-related knowledge on Wikipedia and revealed how citations create a web of scientific knowledge to support coverage of scientific topics like COVID-19 vaccine development.

Understanding how scientific research interacts with the digital knowledge-sphere during the pandemic provides insight into how Wikipedia can facilitate access to science. It also sheds light on how Wikipedia successfully fended of disinformation on the COVID-19 and may provide insight into how its unique model may be deployed in other contexts.

Preprints in motion: tracking changes between posting and journal publication

Authors : Jessica K Polka, Gautam Dey, Máté Pálfy, Federico Nanni, Liam Brierley, Nicholas Fraser, Jonathon Alexis Coates

Amidst the COVID-19 pandemic, preprints in the biomedical sciences are being posted and accessed at unprecedented rates, drawing widespread attention from the general public, press and policymakers for the first time.

This phenomenon has sharpened longstanding questions about the reliability of information shared prior to journal peer review. Does the information shared in preprints typically withstand the scrutiny of peer review, or are conclusions likely to change in the version of record?

We assessed preprints that had been posted and subsequently published in a journal between 1st January and 30th April 2020, representing the initial phase of the pandemic response. We utilised a combination of automatic and manual annotations to quantify how an article changed between the preprinted and published version.

We found that the total number of figure panels and tables changed little between preprint and published articles. Moreover, the conclusions of 6% of non-COVID-19-related and 15% of COVID-19-related abstracts undergo a discrete change by the time of publication, but the majority of these changes do not reverse the main message of the paper.

DOI : https://doi.org/10.1101/2021.02.20.432090

Methodological quality of COVID-19 clinical research

Authors : Richard G. Jung, Pietro Di Santo, Cole Clifford, Graeme Prosperi-Porta, Stephanie Skanes, Annie Hung, Simon Parlow, Sarah Visintini, F. Daniel Ramirez, Trevor Simard & Benjamin Hibbert

The COVID-19 pandemic began in early 2020 with major health consequences. While a need to disseminate information to the medical community and general public was paramount, concerns have been raised regarding the scientific rigor in published reports.

We performed a systematic review to evaluate the methodological quality of currently available COVID-19 studies compared to historical controls. A total of 9895 titles and abstracts were screened and 686 COVID-19 articles were included in the final analysis.

Comparative analysis of COVID-19 to historical articles reveals a shorter time to acceptance (13.0[IQR, 5.0–25.0] days vs. 110.0[IQR, 71.0–156.0] days in COVID-19 and control articles, respectively; p < 0.0001).

Furthermore, methodological quality scores are lower in COVID-19 articles across all study designs. COVID-19 clinical studies have a shorter time to publication and have lower methodological quality scores than control studies in the same journal. These studies should be revisited with the emergence of stronger evidence.

URL : Methodological quality of COVID-19 clinical research

DOI : https://doi.org/10.1038/s41467-021-21220-5

COVID‐19 and the generation of novel scientific knowledge: Evidence‐based decisions and data sharing

Authors : Lucie Perillat, Brian S. Baigrie

Rationale, aims and objectives

The COVID‐19 pandemic has impacted every facet of society, including medical research. This paper is the second part of a series of articles that explore the intricate relationship between the different challenges that have hindered biomedical research and the generation of novel scientific knowledge during the COVID‐19 pandemic.

In the first part of this series, we demonstrated that, in the context of COVID‐19, the scientific community has been faced with numerous challenges with respect to (1) finding and prioritizing relevant research questions and (2) choosing study designs that are appropriate for a time of emergency.

Methods

During the early stages of the pandemic, research conducted on hydroxychloroquine (HCQ) sparked several heated debates with respect to the scientific methods used and the quality of knowledge generated.

Research on HCQ is used as a case study in both papers. The authors explored biomedical databases, peer‐reviewed journals, pre‐print servers and media articles to identify relevant literature on HCQ and COVID‐19, and examined philosophical perspectives on medical research in the context of this pandemic and previous global health challenges.

Results

This second paper demonstrates that a lack of research prioritization and methodological rigour resulted in the generation of fleeting and inconsistent evidence that complicated the development of public health guidelines.

The reporting of scientific findings to the scientific community and general public highlighted the difficulty of finding a balance between accuracy and speed.

Conclusions

The COVID‐19 pandemic presented challenges in terms of (3) evaluating evidence for the purpose of making evidence‐based decisions and (4) sharing scientific findings with the rest of the scientific community.

This second paper demonstrates that the four challenges outlined in the first and second papers have often compounded each other and have contributed to slowing down the creation of novel scientific knowledge during the COVID‐19 pandemic.

DOI : https://doi.org/10.1111/jep.13548

The Pandemic as a Portal: Reimagining Psychological Science as Truly Open and Inclusive

Authors : Alison Ledgerwood, Sa-kiera Hudson, Neil Lewis, Keith Maddox, Cynthia Pickett, Jessica Remedios, Sapna Cheryan, Amanda Diekman, Jin Goh, Stephanie Goodwin, Yuko Munakata, Danielle Navarro, Ivuoma Onyeador, Sanjay Srivastava, Clara Wilkins

Psychological science is at an inflection point: The COVID-19 pandemic has already begun to exacerbate inequalities that stem from our historically closed and exclusive culture. Meanwhile, reform efforts to change the future of our science are too narrow in focus to fully succeed.

In this paper, we call on psychological scientists—focusing specifically on those who use quantitative methods in the United States as one context in which such a conversation can begin—to reimagine our discipline as fundamentally open and inclusive.

First, we discuss who our discipline was designed to serve and how this history produced the inequitable reward and support systems we see today.

Second, we highlight how current institutional responses to address worsening inequalities are inadequate, as well as how our disciplinary perspective may both help and hinder our ability to craft effective solutions.

Third, we take a hard look in the mirror at the disconnect between what we ostensibly value as a field and what we actually practice. Fourth and finally, we lead readers through a roadmap for reimagining psychological science in whatever roles and spaces they occupy, from an informal discussion group in a department to a formal strategic planning retreat at a scientific society.

DOI : https://doi.org/10.31234/osf.io/gdzue

YouTube as a source of information on chloroquine and hydroxychloroquine during the COVID-19 pandemic

Authors : Thales Brandi Ramos, Luciana Castilho Bokehi, Raphael Castilho Bokehi, Taynah da Silava Pinheiro, Erika Barreto de Oliveira, Renan da SilvaGianoti Torres, Jose Raphael Bokehi, Sabrina Calil-Elias, Selma Rodrigues de Castilho

This research aimed to analyze the quality of the information conveyed through YouTube videos in Portuguese on the use of two medicines suggested for the treatment of COVID-19: chloroquine and hydroxychloroquine.

The ‘Brief DISCERN’ questionnaire was applied to assess the quality of the video content as well as baseline characteristics, such as length, views, likes and dislikes, in a total of 90 videos with almost 4,5 million views.

Traditional media accounted for 58,89% of videos. Misleading information was present in most of the videos (63,5%). Despite the ease of access, the videos showed problems in the quality of information.

URL : YouTube as a source of information on chloroquine and hydroxychloroquine during the COVID-19 pandemic

DOI : https://doi.org/10.22323/2.19070206