Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015

Authors : Colin F. Camerer, Anna Dreber, Felix Holzmeister, Teck-Hua Ho, Jürgen Huber, Magnus Johannesson, Michael Kirchler, Gideon Nave, Brian Nosek, Thomas Pfeiffer, Adam Altmejd, Nick Buttrick, Taizan Chan, Yiling Chen, Eskil Forsell, Anup Gampa, Emma Heikensten, Lily Hummer, Taisuke Imai, Siri Isaksson, Dylan Manfredi, Julia Rose, Eric-Jan Wagenmakers, Hang Wu

Being able to replicate scientific findings is crucial for scientific progress. We replicate 21 systematically selected experimental studies in the social sciences published in Nature and Science between 2010 and 2015.

The replications follow analysis plans reviewed by the original authors and pre-registered prior to the replications. The replications are high powered, with sample sizes on average about five times higher than in the original studies.

We find a significant effect in the same direction as the original study for 13 (62%) studies, and the effect size of the replications is on average about 50% of the original effect size. Replicability varies between 12 (57%) and 14 (67%) studies for complementary replicability indicators.

Consistent with these results, the estimated true positive rate is 67% in a Bayesian analysis. The relative effect size of true positives is estimated to be 71%, suggesting that both false positives and inflated effect sizes of true positives contribute to imperfect reproducibility.

Furthermore, we find that peer beliefs of replicability are strongly related to replicability, suggesting that the research community could predict which results would replicate and that failures to replicate were not the result of chance alone.

URL : Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015

DOI : https://doi.org/10.1038/s41562-018-0399-z

Reproducible data citations for computational research

Author : Christian Schulz

The general purpose of a scientific publication is the exchange and spread of knowledge. A publication usually reports a scientific result and tries to convince the reader that it is valid.

With an ever-growing number of papers relying on computational methods that make use of large quantities of data and sophisticated statistical modeling techniques, a textual description of the result is often not enough for a publication to be transparent and reproducible.

While there are efforts to encourage sharing of code and data, we currently lack conventions for linking data sources to a computational result that is stated in the main publication text or used to generate a figure or table.

Thus, here I propose a data citation format that allows for an automatic reproduction of all computations. A data citation consists of a descriptor that refers to the functional program code and the input that generated the result.

The input itself may be a set of other data citations, such that all data transformations, from the original data sources to the final result, are transparently expressed by a directed graph.

Functions can be implemented in a variety of programming languages since data sources are expected to be stored in open and standardized text-based file formats.

A publication is then an online file repository consisting of a Hypertext Markup Language (HTML) document and additional data and code source files, together with a summarization of all data sources, similar to a list of references in a bibliography.

URL : https://arxiv.org/abs/1808.07541

The rent’s too high: Self-archive for fair online publication costs

Authors : Robert T. Thibault, Amanda MacPherson, Stevan Harnad, Amir Raz

The main contributors of scientific knowledge, researchers, generally aim to disseminate their findings far and wide. And yet, publishing companies have largely kept these findings behind a paywall.

With digital publication technology markedly reducing cost, this enduring wall seems disproportionate and unjustified; moreover, it has sparked a topical exchange concerning how to modernize academic publishing.

This discussion, however, seems to focus on how to compensate major publishers for providing open access through a « pay to publish » model, in turn transferring financial burdens from libraries to authors and their funders.

Large publishing companies, including Elsevier, Springer Nature, Wiley, PLoS, and Frontiers, continue to earn exorbitant revenues each year, hundreds of millions of dollars of which now come from processing charges for open-access articles.

A less expensive and equally accessible alternative exists: widespread self-archiving of peer-reviewed articles. All we need is awareness of this alternative and the will to employ it.

URL : https://arxiv.org/abs/1808.06130

Towards a culture of open science and data sharing in health and medical research

Author : Anisa Rowhani-Farid

This thesis investigated the factors that contribute to the cultural shift towards open science and data sharing in health and medical research, with a focus on the role health and medical journals play.

The findings of this research demonstrate that journal data sharing policies are not effective and that journals do not currently provide incentives for sharing.

This study contributed to the movement towards more reproducible research by providing empirical evidence for the strengthening of journal data sharing policies and the adoption of an incentive for open research.

DOI : https://doi.org/10.5204/thesis.eprints.119697

How open is open access research in Library and Information Science?

Authors : Wanyenda Leonard Chilimo, Omwoyo Bosire Onyancha

The study investigates Library and Information Science (LIS) journals that published research articles between 2003 and 2013, which were about open access (OA) and were indexed in LIS databases.

The purpose was to investigate the journals’ OA policies, ascertain the degree to which these policies facilitate OA to publications, and investigate whether such texts are also available as OA. The results show that literature growth in the domain has been significant, with a total of 1,402 articles produced during the eleven years under study.

The OA policies of the fifty-six journals that published the highest number of articles were analysed. The results show that most articles (404; 41%) were published in hybrid journals, whereas 272 (29.7%) appeared in OA journals.

Some 143 (53%) of the articles published in hybrid journals were available as green OA copies. In total, 602 (66%) of all the articles published were available as OA.

The results show that the adoption of OA for research articles on that very subject is somewhat higher than in other fields. The study calls on LIS professionals to be conversant with the OA policies of the various journals that may publish their research.

URL : How open is open access research in Library and Information Science?

Alternative location : http://sajlis.journals.ac.za/pub/article/view/1710

Vanishing industries and the rising monopoly of universities in published research

Authors : Vincent Larivière, Benoit Macaluso, Philippe Mongeon, Kyle Siler, Cassidy R. Sugimoto

Anecdotes abound regarding the decline of basic research in industrial and governmental settings, but very little empirical evidence exists about the phenomenon. This article provides a systematic and historical analysis of the contribution of various institutional sectors to knowledge production at the world and country levels across the past four decades.

It highlights a dramatic decline in the diffusion of basic research by industrial and governmental sectors across all countries—with a corresponding increase in the share from universities—as well as an increase of partnerships between universities and other sectors.

Results also shows an increase in the relative share of industries in applied research, as measured through patents. Such divergence in university and industry research activities may hinder industries’ ability to translate basic knowledge into technological innovation, and could lead to a growing misalignment between doctoral training and future job expectations.

Industries and universities must rethink strategies for partnerships and publishing to maximize scientific progress and to ensure the greatest gains for society.

URL : Vanishing industries and the rising monopoly of universities in published research

DOI : https://doi.org/10.1371/journal.pone.0202120

Statistics on Open Access Books Available through the Directory of Open Access Books

Author : Keita Tsuji

Open Access (OA) books available through the Directory of Open Access Books (DOAB) are investigated and the number of titles, the distribution of subjects, languages, publishers, publication years, licensing patterns, etc., are clarified.

Their chronological changes are also shown. The sample comprised 10,866 OA books, which were available through the DOAB as of February 24, 2018.

The results show that OA books are increasing in number at an accelerating rate. As for distribution of subjects, Social Sciences (« H » in the Library of Congress Classification [LCC] codes), Science (« Q » in LCC) and World History and History of Europe, Asia, Africa, Australia, New Zealand, etc. (« D » in LCC) are the most popular.

As for languages, English, French, and German are the most popular. As for publishers, Frontiers Media SA, Presses universitaires de Rennes, and ANU Press are the most popular.

Many books are newly published ones, but older books, published in or before 1999, also began to be available recently. As for the licensing patterns, « CC by-nc-nd » and « CC by » are the most popular. Considering these tendencies, libraries should begin to utilize OA books.

URL : https://arxiv.org/abs/1808.01541