EN

Open Science and the Hype Cycle

Posted on by Hans Dillaerts

Author : George Strawn

The introduction of a new technology or innovation is often accompanied by “ups and downs” in its fortunes. Gartner Inc. defined a so-called hype cycle to describe a general pattern that many innovations experience: technology trigger, peak of inflated expectations, trough of disillusionment, slope of enlightenment, and plateau of productivity.

This article will compare the ongoing introduction of Open Science (OS) with the hype cycle model and speculate on the relevance of that model to OS. Lest the title of this article mislead the reader, be assured that the author believes that OS should happen and that it will happen.

However, I also believe that the path to OS will be longer than many of us had hoped. I will give a brief history of the today’s “semi-open” science, define what I mean by OS, define the hype cycle and where OS is now on that cycle, and finally speculate what it will take to traverse the cycle and rise to its plateau of productivity (as described by Gartner).

URL : Open Science and the Hype Cycle

DOI : https://doi.org/10.1162/dint_a_00081

EN

The Rigor and Transparency Index Quality Metric for Assessing Biological and Medical Science Methods

Posted on by Hans Dillaerts

Authors : Joe Menke, Martijn Roelandse, Burak Ozyurt, Maryann Martone, Anita Bandrowski

The reproducibility crisis is a multifaceted problem involving ingrained practices within the scientific community. Fortunately, some causes are addressed by the author’s adherence to rigor and reproducibility criteria, implemented via checklists at various journals.

We developed an automated tool (SciScore) that evaluates research articles based on their adherence to key rigor criteria, including NIH criteria and RRIDs, at an unprecedented scale. We show that despite steady improvements, less than half of the scoring criteria, such as blinding or power analysis, are routinely addressed by authors; digging deeper, we examined the influence of specific checklists on average scores.

The average score for a journal in a given year was named the Rigor and Transparency Index (RTI), a new journal quality metric. We compared the RTI with the Journal Impact Factor and found there was no correlation. The RTI can potentially serve as a proxy for methodological quality.

URL : The Rigor and Transparency Index Quality Metric for Assessing Biological and Medical Science Methods

DOI : https://doi.org/10.1016/j.isci.2020.101698

EN

Where Do Early Career Researchers Stand on Open Science Practices? A Survey Within the Max Planck Society

Posted on by Hans Dillaerts

Authors : Daniel Toribio-Flórez, Lukas Anneser, Felipe Nathan deOliveira-Lopes, Martijn Pallandt, Isabell Tunn, Hendrik Windel

Open science (OS) is of paramount importance for the improvement of science worldwide and across research fields. Recent years have witnessed a transition toward open and transparent scientific practices, but there is still a long way to go.

Early career researchers (ECRs) are of crucial relevance in the process of steering toward the standardization of OS practices, as they will become the future decision makers of the institutional change that necessarily accompanies this transition. Thus, it is imperative to gain insight into where ECRs stand on OS practices.

Under this premise, the Open Science group of the Max Planck PhDnet designed and conducted an online survey to assess the stance toward OS practices of doctoral candidates from the Max Planck Society.

As one of the leading scientific institutions for basic research worldwide, the Max Planck Society provides a considerable population of researchers from multiple scientific fields, englobed into three sections: biomedical sciences, chemistry, physics and technology, and human and social sciences.

From an approximate total population of 5,100 doctoral candidates affiliated with the Max Planck Society, the survey collected responses from 568 doctoral candidates. The survey assessed self-reported knowledge, attitudes, and implementation of different OS practices, namely, open access publications, open data, preregistrations, registered reports, and replication studies.

ECRs seemed to hold a generally positive view toward these different practices and to be interested in learning more about them. Furthermore, we found that ECRs’ knowledge and positive attitudes predicted the extent to which they implemented these OS practices, although levels of implementation were rather low in the past. We observed differences and similarities between scientific sections.

We discuss these differences in terms of need and feasibility to apply these OS practices in specific scientific fields, but additionally in relation to the incentive systems that shape scientific communities. Lastly, we discuss the implications that these results can have for the training and career advancement of ECRs, and ultimately, for the consolidation of OS practices.

URL : Where Do Early Career Researchers Stand on Open Science Practices? A Survey Within the Max Planck Society

DOI : https://doi.org/10.3389/frma.2020.586992

EN

Deep Learning in Mining Biological Data

Posted on by Hans Dillaerts

Authors : Mufti Mahmud, M. Shamim Kaiser, T. Martin McGinnity, Amir Hussain

Recent technological advancements in data acquisition tools allowed life scientists to acquire multimodal data from different biological application domains. Categorized in three broad types (i.e. images, signals, and sequences), these data are huge in amount and complex in nature.

Mining such enormous amount of data for pattern recognition is a big challenge and requires sophisticated data-intensive machine learning techniques. Artificial neural network-based learning systems are well known for their pattern recognition capabilities, and lately their deep architectures—known as deep learning (DL)—have been successfully applied to solve many complex pattern recognition problems.

To investigate how DL—especially its different architectures—has contributed and been utilized in the mining of biological data pertaining to those three types, a meta-analysis has been performed and the resulting resources have been critically analysed. Focusing on the use of DL to analyse patterns in data from diverse biological domains, this work investigates different DL architectures’ applications to these data.

This is followed by an exploration of available open access data sources pertaining to the three data types along with popular open-source DL tools applicable to these data. Also, comparative investigations of these tools from qualitative, quantitative, and benchmarking perspectives are provided.

Finally, some open research challenges in using DL to mine biological data are outlined and a number of possible future perspectives are put forward.

URL : Deep Learning in Mining Biological Data

DOI : https://doi.org/10.1007/s12559-020-09773-x

EN

Questionable Research Practices and Open Science in Quantitative Criminology

Posted on by Hans Dillaerts

Authors : Jason Chinn, Justin Pickett, Simine Vazire, Alex Holcombe

Objectives

Questionable research practices (QRPs) lead to incorrect research results and contribute to irreproducibility in science. Researchers and institutions have proposed open science practices (OSPs) to improve the detectability of QRPs and the credibility of science. We examine the prevalence of QRPs and OSPs in criminology, and researchers’ opinions of those practices.

Methods

We administered an anonymous survey to authors of articles published in criminology journals. Respondents self-reported their own use of 10 QRPs and 5 OSPs. They also estimated the prevalence of use by others, and reported their attitudes toward the practices.

Results

QRPs and OSPs are both common in quantitative criminology, about as common as they are in other fields. Criminologists who responded to our survey support using QRPs in some circumstances, but are even more supportive of using OSPs.

We did not detect a significant relationship between methodological training and either QRP or OSP use.

Support for QRPs is negatively and significantly associated with support for OSPs. Perceived prevalence estimates for some practices resembled a uniform distribution, suggesting criminologists have little knowledge of the proportion of researchers that engage in certain questionable practices.

Conclusions

Most quantitative criminologists in our sample use QRPs, and many use multiple QRPs. The substantial prevalence of QRPs raises questions about the validity and reproducibility of published criminological research.

We found promising levels of OSP use, albeit at levels lagging what researchers endorse. The findings thus suggest that additional reforms are needed to decrease QRP use and increase the use of OSPs.

URL : Questionable Research Practices and Open Science in Quantitative Criminology

DOI : https://doi.org/10.31235/osf.io/bwm7s

EN

Cite Unseen: Theory and Evidence on the Effect of Open Access on Cites to Academic Articles Across the Quality Spectrum

Posted on by Hans Dillaerts

Authors : Mark J. McCabe, Christopher Snyder

Our previous paper (McCabe and Snyder 2014) contained the provocative result that, despite a positive average effect, open access reduces cites to some articles, in particular those published in lower-tier journals.

We propose a model in which open access leads more readers to acquire the full text, yielding more cites from some, but fewer cites from those who would have cited the article based on superficial knowledge but who refrain once they learn that the article is a bad match.

We test the theory with data for over 200,000 science articles binned by cites received during a pre-study period. Consistent with the theory, the marginal effect of open access is negative for the least-cited articles, positive for the most cited, and generally monotonic for quality levels in between.

Also consistent with the theory is a magnification of these effects for articles placed on PubMed Central, one of the broadest open-access platforms, and the differential pattern of results for cites from insiders versus outsiders to the article’s field.

URL : https://www.nber.org/papers/w28128

EN

The move to open: medical library leadership in scholarly communication

Posted on by Hans Dillaerts

Author : Chris Shaffer

Over the years, health sciences librarians have been change agents, leading the charge on issues of importance to the profession and the communities we serve. From its founding in 1898 with the Exchange, the Medical Library Association (MLA) has been dedicated to improving access to health information.

In 2003, the Board of Directors published a statement supporting open access to information generated from federally funded scientific and medical research and maintained that having access to timely, relevant, and accurate information is vital to the health of the nation and its education and research programs.

At some financial risk, the association made the Journal of the Medical Library Association (JMLA) open access and published the entire archive of JMLA and its predecessor, the Bulletin of the Medical Library Association, in PubMed Central. Nearly two decades later, the promise of open access and open science finally seems to be coming to fruition.

In the 2020 Janet Doe Lecture, Chris Shaffer, AHIP, described the ways that MLA has led the profession, standing behind a shared vision and “walking the walk.” In challenging listeners to embrace open science, he affirmed that, as leaders in improving access to health sciences information since 1898, medical librarians must work in the open science arena to realize our vision “that quality information is essential for improved health.”

URL : The move to open: medical library leadership in scholarly communication

DOI : https://doi.org/10.5195/jmla.2021.1127