Archives des mots-clés : Isabelle Boutron

Comparison of effect estimates between preprints and peer-reviewed journal articles of COVID-19 trials

Posté le par

Authors : Mauricia Davidson, Theodoros Evrenoglou, Carolina Graña, Anna Chaimani, Isabelle Boutron

Background

Preprints are increasingly used to disseminate research results, providing multiple sources of information for the same study. We assessed the consistency in effect estimates between preprint and subsequent journal article of COVID-19 randomized controlled trials.

Methods

The study utilized data from the COVID-NMA living systematic review of pharmacological treatments for COVID-19 (covid-nma.com) up to July 20, 2022. We identified randomized controlled trials (RCTs) evaluating pharmacological treatments vs. standard of care/placebo for patients with COVID-19 that were originally posted as preprints and subsequently published as journal articles.

Trials that did not report the same analysis in both documents were excluded. Data were extracted independently by pairs of researchers with consensus to resolve disagreements. Effect estimates extracted from the first preprint were compared to effect estimates from the journal article.

Results

The search identified 135 RCTs originally posted as a preprint and subsequently published as a journal article. We excluded 26 RCTs that did not meet the eligibility criteria, of which 13 RCTs reported an interim analysis in the preprint and a final analysis in the journal article. Overall, 109 preprint–article RCTs were included in the analysis.

The median (interquartile range) delay between preprint and journal article was 121 (73–187) days, the median sample size was 150 (71–464) participants, 76% of RCTs had been prospectively registered, 60% received industry or mixed funding, 72% were multicentric trials. The overall risk of bias was rated as ‘some concern’ for 80% of RCTs.

We found that 81 preprint–article pairs of RCTs were consistent for all outcomes reported. There were nine RCTs with at least one outcome with a discrepancy in the number of participants with outcome events or the number of participants analyzed, which yielded a minor change in the estimate of the effect. Furthermore, six RCTs had at least one outcome missing in the journal article and 14 RCTs had at least one outcome added in the journal article compared to the preprint. There was a change in the direction of effect in one RCT. No changes in statistical significance or conclusions were found.

Conclusions

Effect estimates were generally consistent between COVID-19 preprints and subsequent journal articles. The main results and interpretation did not change in any trial. Nevertheless, some outcomes were added and deleted in some journal articles.

URL : Comparison of effect estimates between preprints and peer-reviewed journal articles of COVID-19 trials

DOI : https://doi.org/10.1186/s12874-023-02136-8

Day-to-day discovery of preprint–publication links

Posté le par

Authors : Guillaume Cabanac, Theodora Oikonomidi, Isabelle Boutron

Preprints promote the open and fast communication of non-peer reviewed work. Once a preprint is published in a peer-reviewed venue, the preprint server updates its web page: a prominent hyperlink leading to the newly published work is added.

Linking preprints to publications is of utmost importance as it provides readers with the latest version of a now certified work. Yet leading preprint servers fail to identify all existing preprint–publication links.

This limitation calls for a more thorough approach to this critical information retrieval task: overlooking published evidence translates into partial and even inaccurate systematic reviews on health-related issues, for instance.

We designed an algorithm leveraging the Crossref public and free source of bibliographic metadata to comb the literature for preprint–publication links. We tested it on a reference preprint set identified and curated for a living systematic review on interventions for preventing and treating COVID-19 performed by international collaboration: the COVID-NMA initiative (covid-nma.com).

The reference set comprised 343 preprints, 121 of which appeared as a publication in a peer-reviewed journal. While the preprint servers identified 39.7% of the preprint–publication links, our linker identified 90.9% of the expected links with no clues taken from the preprint servers.

The accuracy of the proposed linker is 91.5% on this reference set, with 90.9% sensitivity and 91.9% specificity. This is a 16.26% increase in accuracy compared to that of preprint servers. We release this software as supplementary material to foster its integration into preprint servers’ workflows and enhance a daily preprint–publication chase that is useful to all readers, including systematic reviewers.

This preprint–publication linker currently provides day-to-day updates to the biomedical experts of the COVID-NMA initiative.

URL : Day-to-day discovery of preprint–publication links

DOI : https://doi.org/10.1007/s11192-021-03900-7

Influence and management of conflicts of interest in randomised clinical trials: qualitative interview study

Posté le par

Authors : Lasse Østengaard, Andreas Lundh, Tine Tjørnhøj-Thomsen, Suhayb Abdi, Mustafe H A Gelle, Lesley A Stewart, Isabelle Boutron, Asbjørn Hróbjartsson

Objective

To characterise and analyse the experiences of trial researchers of if and how conflicts of interest had unduly influenced clinical trials they had worked on, what management strategies they had used to minimise any potential influence, and their experiences and views on conflicts of interest more generally.

Design

Qualitative interview study.

Participants

Trial researchers who had participated in at least 10 clinical trials with methodological or statistical expertise. Researchers differed by geographical location, educational background, and experience with different types of funders. Interviewees were identified by searches on Web of Science and snowball sampling. 52 trial researchers were approached by email; 20 agreed to be interviewed.

Setting

Interviews conducted by telephone, recorded, transcribed verbatim, imported to NVivo 12, and analysed by systematic text condensation. Semistructured interviews focused on financial and non-financial conflicts of interest.

Results

The interviewees had participated in a median of 37.5 trials and were mainly male physicians who had experience with commercial and non-commercial trial funders. Two predefined themes (influence of conflicts of interest and management strategies) and two additional themes (definition and reporting of conflicts of interest) emerged.

Examples of perceived influence of conflicts of interest were: choice of inferior comparator, manipulation of the randomisation process, prematurely stopping the trials, fabrication of data, blocking access to data, and spin (eg, overly favourable interpretation of the results).

Examples of strategies to manage conflicts of interest were: disclosure procedures, exclusion of the funder from design and analysis, independent committees, contracts ensuring complete access to the data, and no restriction by the funder on analysis and reporting.

Interviewees used different definitions or thresholds for what they considered to be conflicts of interest, and they described different criteria for when to report them. Some interviewees considered non-commercial financial conflicts of interest (eg, funding of trials by governmental health agencies with a political agenda) to be equally or more important than commercial financial conflicts of interest (eg, funding by drug and device companies), but more challenging to report and manage.

Conclusion

This study described how trial researchers perceive conflicts of interest unduly influencing clinical trials they had worked on, and the management strategies they used to prevent these influences.

The results indicated considerable variability in researchers’ understanding of what conflicts of interest are and when they should be reported.

URL : Influence and management of conflicts of interest in randomised clinical trials: qualitative interview study

DOI : https://doi.org/10.1136/bmj.m3764

A systematic examination of preprint platforms for use in the medical and biomedical sciences setting

Posté le par

Authors : Jamie J Kirkham, Naomi Penfold, Fiona Murphy, Isabelle Boutron, John PA Ioannidis, Jessica K Polka, David Moher

Objectives

The objective of this review is to identify all preprint platforms with biomedical and medical scope and to compare and contrast the key characteristics and policies of these platforms. We also aim to provide a searchable database to enable relevant stakeholders to compare between platforms.

Study Design and Setting

Preprint platforms that were launched up to 25th June 2019 and have a biomedical and medical scope according to MEDLINE’s journal selection criteria were identified using existing lists, web-based searches and the expertise of both academic and non-academic publication scientists.

A data extraction form was developed, pilot-tested and used to collect data from each preprint platform’s webpage(s). Data collected were in relation to scope and ownership; content-specific characteristics and information relating to submission, journal transfer options, and external discoverability; screening, moderation, and permanence of content; usage metrics and metadata.

Where possible, all online data were verified by the platform owner or representative by correspondence.

Results

A total of 44 preprint platforms were identified as having biomedical and medical scope, 17 (39%) were hosted by the Open Science Framework preprint infrastructure, six (14%) were provided by F1000 Research Ltd (the Open Research Central infrastructure) and 21 (48%) were other independent preprint platforms. Preprint platforms were either owned by non-profit academic groups, scientific societies or funding organisations (n=28; 64%), owned/partly owned by for-profit publishers or companies (n=14; 32%) or owned by individuals/small communities (n=2; 5%).

Twenty-four (55%) preprint platforms accepted content from all scientific fields although some of these had restrictions relating to funding source, geographical region or an affiliated journal’s remit.

Thirty-three (75%) preprint platforms provided details about article screening (basic checks) and 14 (32%) of these actively involved researchers with context expertise in the screening process.

The three most common screening checks related to the scope of the article, plagiarism and legal/ethical/societal issues and compliance. Almost all preprint platforms allow submission to any peer-reviewed journal following publication, have a preservation plan for read-access, and most have a policy regarding reasons for retraction and the sustainability of the service.

Forty-one (93%) platforms currently have usage metrics, with the most common metric being the number of downloads presented on the abstract page.

Conclusion

A large number of preprint platforms exist for use in biomedical and medical sciences, all of which offer researchers an opportunity to rapidly disseminate their research findings onto an open-access public server, subject to scope and eligibility.

However, the process by which content is screened before online posting and withdrawn or removed after posting varies between platforms, which may be associated with platform operation, ownership, governance and financing.

DOI : https://doi.org/10.1101/2020.04.27.063578