An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014–2017)

Authors : Tom E. Hardwicke, Joshua D. Wallach, Mallory C. Kidwell, Theiss Bendixen, Sophia Crüwell, John P. A. Ioannidis

Serious concerns about research quality have catalysed a number of reform initiatives intended to improve transparency and reproducibility and thus facilitate self-correction, increase efficiency and enhance research credibility.

Meta-research has evaluated the merits of some individual initiatives; however, this may not capture broader trends reflecting the cumulative contribution of these efforts.

In this study, we manually examined a random sample of 250 articles in order to estimate the prevalence of a range of transparency and reproducibility-related indicators in the social sciences literature published between 2014 and 2017.

Few articles indicated availability of materials (16/151, 11% [95% confidence interval, 7% to 16%]), protocols (0/156, 0% [0% to 1%]), raw data (11/156, 7% [2% to 13%]) or analysis scripts (2/156, 1% [0% to 3%]), and no studies were pre-registered (0/156, 0% [0% to 1%]).

Some articles explicitly disclosed funding sources (or lack of; 74/236, 31% [25% to 37%]) and some declared no conflicts of interest (36/236, 15% [11% to 20%]). Replication studies were rare (2/156, 1% [0% to 3%]).

Few studies were included in evidence synthesis via systematic review (17/151, 11% [7% to 16%]) or meta-analysis (2/151, 1% [0% to 3%]). Less than half the articles were publicly available (101/250, 40% [34% to 47%]).

Minimal adoption of transparency and reproducibility-related research practices could be undermining the credibility and efficiency of social science research. The present study establishes a baseline that can be revisited in the future to assess progress.

URL : An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014–2017)

DOI : https://doi.org/10.1098/rsos.190806

A Realistic Guide to Making Data Available Alongside Code to Improve Reproducibility

Authors : Nicholas J Tierney, Karthik Ram

Data makes science possible. Sharing data improves visibility, and makes the research process transparent. This increases trust in the work, and allows for independent reproduction of results.

However, a large proportion of data from published research is often only available to the original authors. Despite the obvious benefits of sharing data, and scientists’ advocating for the importance of sharing data, most advice on sharing data discusses its broader benefits, rather than the practical considerations of sharing.

This paper provides practical, actionable advice on how to actually share data alongside research. The key message is sharing data falls on a continuum, and entering it should come with minimal barriers.

URL : https://arxiv.org/abs/2002.11626

Rethinking the Journal Impact Factor and Publishing in the Digital Age

Authors : Mark S. Nestor, Daniel Fischer, David Arnold, Brian Berman, James Q. Del Rosso

Clinical and experimental literature search has changed significantly over the past few decades, and with it, the way in which we value information. Today, our need for immediate access to relevant and specific literature, regardless of specialty, has led to a growing demand for open access to publications.

The Journal Impact Factor (JIF) has been a long-time standard for representing the quality or “prestige” of a journal, but it appears to be losing its relevance. Here, we define the JIF and deconstruct its validity as a modern measure of a journal’s quality, discuss the current models of academic publication, including their advantages and shortcomings, and discuss the benefits and shortcomings of a variety of open-access models, including costs to the author.

We have quantified a nonsubscribed physician’s access to full articles associated with dermatologic disease and aesthetics cited on PubMed. For some of the most common dermatology conditions, 23.1 percent of citations (ranging from 17.2% for melasma to 31.9% for malignant melanoma) were available as free full articles, and for aesthetic procedures, 18.9 percent of citations (ranging from 11.9% for laser hair removal to 27.9% for botulinum toxin) were available as free full articles.

Finally, we discuss existing alternative metrics for measuring journal impact and propose the adoption of a superior publishing model, one that satisfies modern day standards of scholarly knowledge pursuit and dissemination of scholarly publications for dermatology and all of medical science.

URL : https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7028381/

Funding Sources for Open Access Article Processing Charges in the Social Sciences, Arts, and Humanities in the United States

Authors : Melissa H. Cantrell, Juleah A. Swanson

Article processing charges (APCs) are one method of many to ensure open access to research literature, but studies that explore the funding sources for such payments, especially as related to open access publications in the arts, humanities, and social sciences, have been limited.

This study seeks to understand the range of funding sources that are available and used by faculties in these disciplines to pay for APCs associated with publishing in open access journals, as well as attitudes towards and awareness of available institutional funds that may inflect future engagement with open access publishing.

The authors distributed a survey to faculty who had an open access journal article published in 2017 from three doctoral granting, high research activity universities in the United States.

Twenty-two scholars participated in the final survey, ten of whom indicated that they paid an APC for their publication. While the results cannot make generalizations about funding sources, they do suggest that both the prevalence of APCs as well as attitudes about open access engagement may be influenced by disciplinary self-identification.

This research contributes to discussions around the future of open access funding models as well as to disciplinary outreach regarding APC funding for journal publications.

URL : Funding Sources for Open Access Article Processing Charges in the Social Sciences, Arts, and Humanities in the United States

DOI : https://doi.org/10.3390/publications8010012

Canadian OA scholarly journals: An exhaustive survey

Author : Marc Couture

This report presents the results of an exhaustive study of more than 500 active, legitimate, Canadian, Open Access scholarly journals.

After an extensive discussion on the definition chosen for each of these terms, which determines the number of journals retained for the study, I present various characteristics of these journals, followed by a discussion on the issues faced by journals not indexed in DOAJ that would consider to apply.

I present next the results of a detailed investigation on the way these journals manage copyright, and the various problems I detected in this regard.

URL : https://r-libre.teluq.ca/106/

Preprints and Scholarly Communication: An Exploratory Qualitative Study of Adoption, Practices, Drivers and Barriers

Authors : Andrea Chiarelli, Rob Johnson, Stephen Pinfield, Emma Richens

Background

Since 2013, there has been a dramatic increase in the number of preprint servers. Little is known about the position of researchers, funders, research performing organisations and other stakeholders with respect to this fast-paced landscape.

In this article, we explore the perceived benefits and challenges of preprint posting, alongside issues including infrastructure and financial sustainability. We also discuss the definition of a ‘preprint’ in different communities, and the impact this has on uptake.

Methods

This study is based on 38 semi-structured interviews of key stakeholders, based on a purposive heterogeneous sampling approach and undertaken between October 2018 and January 2019.

Interviewees were primarily drawn from biology, chemistry and psychology, where use of preprints is growing. Interviews were recorded, transcribed and subjected to thematic analysis to identify trends. Interview questions were designed based on Innovation Diffusion Theory, which was also used to interpret our results.

Results

Participants were conscious of the rising prominence of preprints and cited early and fast dissemination as their most appealing feature. Preprints were also considered to enable broader access to scientific literature and increased opportunities for informal commenting.

The main concerns related to the lack of quality assurance and the ‘Ingelfinger rule’. We identified trust as an essential factor in preprint posting, and highlight the enabling role of Twitter in showcasing preprints.

Conclusions

The preprints landscape is evolving fast, and disciplinary communities are at different stages in the innovation diffusion process. The landscape is characterised by experimentation, which leads to the conclusion that a one-size-fits-all approach to preprints is not feasible.

Cooperation and active engagement between the stakeholders involved will play an important role going forward. We share questions for the further development of the preprints landscape, with the most important being whether preprint posting will develop as a publisher- or researcher-centric practice.

URL : Preprints and Scholarly Communication: An Exploratory Qualitative Study of Adoption, Practices, Drivers and Barriers

DOI : https://doi.org/10.12688/f1000research.19619.2