Survey on Scientific Shared Resource Rigor and Reproducibility

Authors : Kevin L. Knudtson, Robert H. Carnahan, Rebecca L. Hegstad-Davies, Nancy C. Fisher, Belynda Hicks, Peter A. Lopez, Susan M. Meyn, Sheenah M. Mische, Frances Weis-Garcia, Lisa D. White, Katia Sol-Church

Shared scientific resources, also known as core facilities, support a significant portion of the research conducted at biomolecular research institutions.

The Association of Biomolecular Resource Facilities (ABRF) established the Committee on Core Rigor and Reproducibility (CCoRRe) to further its mission of integrating advanced technologies, education, and communication in the operations of shared scientific resources in support of reproducible research.

In order to first assess the needs of the scientific shared resource community, the CCoRRe solicited feedback from ABRF members via a survey. The purpose of the survey was to gain information on how U.S. National Institutes of Health (NIH) initiatives on advancing scientific rigor and reproducibility influenced current services and new technology development.

In addition, the survey aimed to identify the challenges and opportunities related to implementation of new reporting requirements and to identify new practices and resources needed to ensure rigorous research.

The results revealed a surprising unfamiliarity with the NIH guidelines. Many of the perceived challenges to the effective implementation of best practices (i.e., those designed to ensure rigor and reproducibility) were similarly noted as a challenge to effective provision of support services in a core setting. Further, most cores routinely use best practices and offer services that support rigor and reproducibility.

These services include access to well-maintained instrumentation and training on experimental design and data analysis as well as data management. Feedback from this survey will enable the ABRF to build better educational resources and share critical best-practice guidelines.

These resources will become important tools to the core community and the researchers they serve to impact rigor and transparency across the range of science and technology.

DOI : https://dx.doi.org/10.7171%2Fjbt.19-3003-001

Public Views on Models for Accessing Genomic and Health Data for Research: Mixed Methods Study

Authors : Kerina H Jones, Helen Daniels, Emma Squires, David V Ford

Background

The literature abounds with increasing numbers of research studies using genomic data in combination with health data (eg, health records and phenotypic and lifestyle data), with great potential for large-scale research and precision medicine.

However, concerns have been raised about social acceptability and risks posed for individuals and their kin. Although there has been public engagement on various aspects of this topic, there is a lack of information about public views on data access models.

Objective

This study aimed to address the lack of information on the social acceptability of access models for reusing genomic data collected for research in conjunction with health data.

Models considered were open web-based access, released externally to researchers, and access within a data safe haven.

Methods

Views were ascertained using a series of 8 public workshops (N=116). The workshops included an explanation of benefits and risks in using genomic data with health data, a facilitated discussion, and an exit questionnaire.

The resulting quantitative data were analyzed using descriptive and inferential statistics, and the qualitative data were analyzed for emerging themes.

Results

Respondents placed a high value on the reuse of genomic data but raised concerns including data misuse, information governance, and discrimination. They showed a preference for giving consent and use of data within a safe haven over external release or open access.

Perceived risks with open access included data being used by unscrupulous parties, with external release included data security, and with safe havens included the need for robust safeguards.

Conclusions: This is the first known study exploring public views of access models for reusing anonymized genomic and health data in research.

It indicated that people are generally amenable but prefer data safe havens because of perceived sensitivities. We recommend that public views be incorporated into guidance on models for the reuse of genomic and health data.

URL : Public Views on Models for Accessing Genomic and Health Data for Research: Mixed Methods Study

DOI : https://doi.org/10.2196/14384

Publication in a medical student journal predicts short- and long-term academic success: a matched-cohort study

Authors : Ibrahim S. Al-Busaidi, Cameron I. Wells, Tim J. Wilkinson

Background

Medical student journals play a critical role in promoting academic research and publishing amongst medical students, but their impact on students’ future academic achievements has not been examined.

We aimed to evaluate the short- and long-term effects of publication in the New Zealand Medical Student Journal (NZMSJ) through examining rates of post-graduation publication, completion of higher academic degrees, and pursuing an academic career.

Methods

Student-authored original research publications in the NZMSJ during the period 2004–2011 were retrospectively identified. Gender-, university- and graduation year-matched controls were identified from publicly available databases in a 2:1 ratio (two controls for each student authors).

Date of graduation, current clinical scope of practice, completion of higher academic degrees, and attainment of an academic position for both groups were obtained from Google searches, New Zealand graduate databases, online lists of registered doctors in New Zealand and Australia, and author affiliation information from published articles.

Pre- and post-graduation PubMed®-indexed publications were identified using standardised search criteria.

Results

Fifty publications authored by 49 unique students were identified. The median follow-up period after graduation was 7.0 years (range 2–12 years). Compared with controls, studentauthors were significantly more likely to publish in PubMed®-indexed journals (OR 3.09, p = 0.001), obtain a PhD (OR 9.21, p = 0.004) or any higher degree (OR 2.63, p = 0.007), and attain academic positions (OR 2.90, p = 0.047) following graduation.

Conclusion

Publication in a medical student journal is associated with future academic achievement and contributes to develop a clinical academic workforce. Future work should aim to explore motivators and barriers associated with these findings.

URL : Publication in a medical student journal predicts short- and long-term academic success: a matched-cohort study

 

Implementing the FAIR Data Principles in precision oncology: review of supporting initiatives

Authors : Charles Vesteghem, Rasmus Froberg Brøndum, Mads Sønderkær, Mia Sommer, Alexander Schmitz, Julie Støve Bødker, Karen Dybkær, Tarec Christoffer El-Galaly, Martin Bøgsted

Compelling research has recently shown that cancer is so heterogeneous that single research centres cannot produce enough data to fit prognostic and predictive models of sufficient accuracy. Data sharing in precision oncology is therefore of utmost importance.

The Findable, Accessible, Interoperable and Reusable (FAIR) Data Principles have been developed to define good practices in data sharing. Motivated by the ambition of applying the FAIR Data Principles to our own clinical precision oncology implementations and research, we have performed a systematic literature review of potentially relevant initiatives.

For clinical data, we suggest using the Genomic Data Commons model as a reference as it provides a field-tested and well-documented solution. Regarding classification of diagnosis, morphology and topography and drugs, we chose to follow the World Health Organization standards, i.e. ICD10, ICD-O-3 and Anatomical Therapeutic Chemical classifications, respectively.

For the bioinformatics pipeline, the Genome Analysis ToolKit Best Practices using Docker containers offer a coherent solution and have therefore been selected. Regarding the naming of variants, we follow the Human Genome Variation Society’s standard.

For the IT infrastructure, we have built a centralized solution to participate in data sharing through federated solutions such as the Beacon Networks.

URL : Implementing the FAIR Data Principles in precision oncology: review of supporting initiatives

DOI : https://doi.org/10.1093/bib/bbz044

The effect of bioRxiv preprints on citations and altmetrics

Authors : Nicholas Fraser, Fakhri Momeni, Philipp Mayr, Isabella Peters

A potential motivation for scientists to deposit their scientific work as preprints is to enhance its citation or social impact, an effect which has been empirically observed for preprints in physics, astronomy and mathematics deposited to arXiv. In this study we assessed the citation and altmetric advantage of bioRxiv, a preprint server for the biological sciences.

We retrieved metadata of all bioRxiv preprints deposited between November 2013 and December 2017, and matched them to articles that were subsequently published in peer-reviewed journals. Citation data from Scopus and altmetric data from Altmetric.com were used to compare citation and online sharing behaviour of bioRxiv preprints, their related journal articles, and non-deposited articles published in the same journals.

We found that bioRxiv-deposited journal articles received a sizeable citation and altmetric advantage over non-deposited articles. Regression analysis reveals that this advantage is not explained by multiple explanatory variables related to the article and its authorship.

bioRxiv preprints themselves are being directly cited in journal articles, regardless of whether the preprint has been subsequently published in a journal. bioRxiv preprints are also shared widely on Twitter and in blogs, but remain relatively scarce in mainstream media and Wikipedia articles, in comparison to peer-reviewed journal articles.

URL : https://www.biorxiv.org/content/10.1101/673665v1

Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature

Authors : Clarissa F. D. Carneiro, Victor G. S. Queiroz, Thiago C. Moulin, Carlos A. M. Carvalho, Clarissa B. Haas, Danielle Rayêe, David E. Henshall, Evandro A. De-Souza, Felippe Espinelli, Flávia Z. Boos, Gerson D. Guercio, Igor R. Costa, Karina L. Hajdu, Martin Modrák, Pedro B. Tan, Steven J. Burgess, Sylvia F. S. Guerra, Vanessa T. Bortoluzzi, Olavo B. Amara

Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader’s ability to independently interpret data and reproduce findings.

In this observational study, we compared random samples of articles published in bioRxiv and in PubMed-indexed journals in 2016 using a quality of reporting questionnaire. We found that peer-reviewed articles had, on average, higher quality of reporting than preprints, although this difference was small.

We found larger differences favoring PubMed in subjective ratings of how clearly titles and abstracts presented the main findings and how easy it was to locate relevant reporting information.

Interestingly, an exploratory analysis showed that preprints with figures and legends embedded within text had reporting scores similar to PubMed articles.

These differences cannot be directly attributed to peer review or editorial processes, as manuscripts might already differ before submission due to greater uptake of preprints by particular research communities.

Nevertheless, our results show that quality of reporting in preprints in the life sciences is within a similar range as that of peer-reviewed articles, albeit slightly lower on average, supporting the idea that preprints should be considered valid scientific contributions.

An ongoing second phase of the project is comparing preprints to their own published versions in order to more directly assess the effects of peer review.

URL : Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature

DOI : https://doi.org/10.1101/581892

Open access policies of leading medical journals: a cross-sectional study

Authors : Tim S Ellison, Laura Schmidt, Amy Williams, Christopher C Winchester

Objectives

Academical and not-for-profit research funders are increasingly requiring that the research they fund must be published open access, with some insisting on publishing with a Creative Commons Attribution (CC BY) licence to allow the broadest possible use.

We aimed to clarify the open access variants provided by leading medical journals and record the availability of the CC BY licence for commercially funded research.

Methods

We identified medical journals with a 2015 impact factor of ≥15.0 on 24 May 2017, then excluded from the analysis journals that only publish review articles. Between 29 June 2017 and 26 July 2017, we collected information about each journal’s open access policies from their websites and/or by email contact.

We contacted the journals by email again between 6 December 2017 and 2 January 2018 to confirm our findings.

Results

Thirty-five medical journals publishing original research from 13 publishers were included in the analysis. All 35 journals offered some form of open access allowing articles to be free-to-read, either immediately on publication or after a delay of up to 12 months.

Of these journals, 21 (60%) provided immediate open access with a CC BY licence under certain circumstances (eg, to specific research funders). Of these 21, 20 only offered a CC BY licence to authors funded by non-commercial organisations and one offered this option to any funder who required it.

Conclusions

Most leading medical journals do not offer to authors reporting commercially funded research an open access licence that allows unrestricted sharing and adaptation of the published material.

The journals’ policies are therefore not aligned with open access declarations and guidelines. Commercial research funders lag behind academical funders in the development of mandatory open access policies, and it is time for them to work with publishers to advance the dissemination of the research they fund.

URL : Open access policies of leading medical journals: a cross-sectional study

DOI : http://dx.doi.org/10.1136/bmjopen-2018-028655