Étiquette : reproducibility of research

Publié le

Reading the fine print: A review and analysis of business journals’ data sharing policies

Authors : Brianne Dosch, Tyler Martindale

Business librarians offer many data services to their researchers. These services are often focused more on discovery, visualization, and analysis than general data management. But, with the replication crisis facing many business disciplines, there is a need for business librarians to offer more data sharing and general data management support to their researchers.

To find evidence of this data need, 146 business journal’s data sharing policies were reviewed and analyzed to uncover meaningful trends in business research. Results of the study indicate data sharing is not mandated by business journals.

However, data sharing is often encouraged and recommended. This journal policy content analysis provides evidence that business researchers have opportunities to share their research data, and with the right data management support, business librarians can play a significant role in improving the data sharing behaviors of business researchers.

DOI : https://doi.org/10.1080/08963568.2020.1847549

Publié le

Metascience as a scientific social movement

Authors : David Peterson, Aaron Panofsky

Emerging out of the “reproducibility crisis” in science, metascientists have become central players in debates about research integrity, scholarly communication, and science policy. The goal of this article is to introduce metascience to STS scholars, detail the scientific ideology that is apparent in its articles, strategy statements, and research projects, and discuss its institutional and intellectual future.

Put simply, metascience is a scientific social movement that seeks to use the tools of science- especially, quantification and experimentation- to diagnose problems in research practice and improve efficiency.

It draws together data scientists, experimental and statistical methodologists, and open science activists into a project with both intellectual and policy dimensions. Metascientists have been remarkably successful at winning grants, motivating news coverage, and changing policies at science agencies, journals, and universities.

Moreover, metascience represents the apotheosis of several trends in research practice, scientific communication, and science governance including increased attention to methodological and statistical criticism of scientific practice, the promotion of “open science” by science funders and journals, the growing importance of both preprint and data repositories for scientific communication, and the new prominence of data scientists as research makes a turn toward Big Science.

URL : Metascience as a scientific social movement

DOI : https://doi.org/10.31235/osf.io/4dsqa

Publié le

Two years into the Brazilian Reproducibility Initiative: reflections on conducting a large-scale replication of Brazilian biomedical science

Author : Kleber Neves, Clarissa FD Carneiro, Ana Paula Wasilewska-Sampaio, Mariana Abreu, Bruna Valério-Gomes, Pedro B Tan, Olavo B Amaral

Scientists have increasingly recognised that low methodological and analytical rigour combined with publish-or-perish incentives can make the published scientific literature unreliable.

As a response to this, large-scale systematic replications of the literature have emerged as a way to assess the problem empirically. The Brazilian Reproducibility Initiative is one such effort, aimed at estimating the reproducibility of Brazilian biomedical research.

Its goal is to perform multicentre replications of a quasi-random sample of at least 60 experiments from Brazilian articles published over a 20-year period, using a set of common laboratory methods.

In this article, we describe the challenges of managing a multicentre project with collaborating teams across the country, as well as its successes and failures over the first two years.

We end with a brief discussion of the Initiative’s current status and its possible future contributions after the project is concluded in 2021.

URL : Two years into the Brazilian Reproducibility Initiative: reflections on conducting a large-scale replication of Brazilian biomedical science

DOI : https://doi.org/10.1590/0074-02760200328

Publié le

What senior academics can do to support reproducible and open research: a short, three-step guide

Authors : Olivia Kowalczyk, Alexandra Lautarescu, Elisabet Blok, Lorenza Dall’Aglio, Samuel Westwood

Increasingly, policies are being introduced to reward and recognise open research practices, while the adoption of such practices into research routines is being facilitated by many grassroots initiatives.

However, despite this widespread endorsement and support, open research is yet to be widely adopted, with early career researchers being the notable exception. For open research to become the norm, initiatives should engage academics from all career stages, particularly senior academics (namely senior lecturers, readers, professors) given their routine involvement in determining the quality of research.

Senior academics, however, face unique challenges in implementing policy change and supporting grassroots initiatives. Given that – like all researchers – senior academics are in part motivated by self-interest, this paper lays out three feasible steps that senior academics can take to improve the quality and productivity of their research, that also serve to engender open research.

These steps include a) change hiring criteria, b) change how scholarly outputs are credited, and c) change to funding and publishing with open research. The guidance we provide is accompanied by live, crowd-sourced material for further reading.

URL : What senior academics can do to support reproducible and open research: a short, three-step guide

Original location : https://psyarxiv.com/jyfr7

Publié le

Low availability of code in ecology: A call for urgent action

Authors : Antica Culina, Ilona van den Berg, Simon Evans, Alfredo Sánchez-Tójar

Access to analytical code is essential for transparent and reproducible research. We review the state of code availability in ecology using a random sample of 346 nonmolecular articles published between 2015 and 2019 under mandatory or encouraged code-sharing policies.

Our results call for urgent action to increase code availability: only 27% of eligible articles were accompanied by code. In contrast, data were available for 79% of eligible articles, highlighting that code availability is an important limiting factor for computational reproducibility in ecology.

Although the percentage of ecological journals with mandatory or encouraged code-sharing policies has increased considerably, from 15% in 2015 to 75% in 2020, our results show that code-sharing policies are not adhered to by most authors.

We hope these results will encourage journals, institutions, funding agencies, and researchers to address this alarming situation.

URL : Low availability of code in ecology: A call for urgent action

DOI : https://doi.org/10.1371/journal.pbio.3000763

Publié le

The role of metadata in reproducible computational research

Authors : Jeremy Leipzig, Daniel Nüst, Charles Tapley Hoyt, Stian Soiland-Reyes, Karthik Ram, Jane Greenberg

Reproducible computational research (RCR) is the keystone of the scientific method for in silico analyses, packaging the transformation of raw data to published results.

In addition to its role in research integrity, RCR has the capacity to significantly accelerate evaluation and reuse. This potential and wide-support for the FAIR principles have motivated interest in metadata standards supporting RCR.

Metadata provides context and provenance to raw data and methods and is essential to both discovery and validation. Despite this shared connection with scientific data, few studies have explicitly described the relationship between metadata and RCR.

This article employs a functional content analysis to identify metadata standards that support RCR functions across an analytic stack consisting of input data, tools, notebooks, pipelines, and publications.

Our article provides background context, explores gaps, and discovers component trends of embeddedness and methodology weight from which we derive recommendations for future work.

URL : The role of metadata in reproducible computational research

Original location : https://arxiv.org/abs/2006.08589

Publié le

No raw data, no science: another possible source of the reproducibility crisis

Author : Tsuyoshi Miyakawa

A reproducibility crisis is a situation where many scientific studies cannot be reproduced. Inappropriate practices of science, such as HARKing, p-hacking, and selective reporting of positive results, have been suggested as causes of irreproducibility. In this editorial, I propose that a lack of raw data or data fabrication is another possible cause of irreproducibility.

As an Editor-in-Chief of Molecular Brain, I have handled 180 manuscripts since early 2017 and have made 41 editorial decisions categorized as “Revise before review,” requesting that the authors provide raw data.

Surprisingly, among those 41 manuscripts, 21 were withdrawn without providing raw data, indicating that requiring raw data drove away more than half of the manuscripts. I rejected 19 out of the remaining 20 manuscripts because of insufficient raw data.

Thus, more than 97% of the 41 manuscripts did not present the raw data supporting their results when requested by an editor, suggesting a possibility that the raw data did not exist from the beginning, at least in some portions of these cases.

Considering that any scientific study should be based on raw data, and that data storage space should no longer be a challenge, journals, in principle, should try to have their authors publicize raw data in a public database or journal site upon the publication of the paper to increase reproducibility of the published results and to increase public trust in science.

URL : No raw data, no science: another possible source of the reproducibility crisis

DOI : https://doi.org/10.1186/s13041-020-0552-2