Replication studies in economics—How many and which papers are chosen for replication, and why?

Authors : Frank Mueller-Langer, Benedikt Fecher, Dietmar Harhoff, Gert G.Wagner

We investigate how often replication studies are published in empirical economics and what types of journal articles are replicated. We find that between 1974 and 2014 0.1% of publications in the top 50 economics journals were replication studies.

We consider the results of published formal replication studies (whether they are negating or reinforcing) and their extent: Narrow replication studies are typically devoted to mere replication of prior work, while scientific replication studies provide a broader analysis.

We find evidence that higher-impact articles and articles by authors from leading institutions are more likely to be replicated, whereas the replication probability is lower for articles that appeared in top 5 economics journals.

Our analysis also suggests that mandatory data disclosure policies may have a positive effect on the incidence of replication.

URL : Replication studies in economics—How many and which papers are chosen for replication, and why?

DOI : https://doi.org/10.1016/j.respol.2018.07.019

From Open Access to Open Data: collaborative work in the university libraries of Catalonia

Authors: Mireia Alcalá Ponce de León, Lluís Anglada i de Ferrer

In the last years, the scientific community and funding bodies have paid attention to collected, generated or used data throughout different research activities. The dissemination of these data becomes one of the constituent elements of Open Science.

For this reason, many funders are requiring or promoting the development of Data Management Plans, and depositing open data following the FAIR principles (Findable, Accessible, Interoperable and Reusable).

Libraries and research offices of Catalan universities –which coordinately work within the Open Science Area of CSUC– offer support services to research data management. The different works carried out at the Consortium level will be presented, as well the implementation of the service in each university.

URL : From Open Access to Open Data: collaborative work in the university libraries of Catalonia

DOI : http://doi.org/10.18352/lq.10253

Wikipedia: an opportunity to rethink the links between sources’ credibility, trust and authority

Authors : Gilles Sahut, André Tricot

The Web and its main tools (Google, Wikipedia, Facebook, Twitter) deeply raise and renew fundamental questions, that everyone asks almost every day: Is this information or content true? Can I trust this author or source?

These questions are not new, they have been the same with books, newspapers, broadcasting and television, and, more fundamentally, in every human interpersonal communication.

This paper is focused on two scientific problems on this issue. The first one is theoretical: to address this issue, many concepts have been used in library and information sciences, communication and psychology.

The links between these concepts are not clear: sometimes two concepts are considered as synonymous, sometimes as very different. The second one is historical: sources like Wikipedia deeply challenge the epistemic evaluation of information sources, compared to previous modes of information production.

This paper proposes an integrated and simple model considering the relation between a user, a document and an author as human communication. It reduces the problem to three concepts: credibility as a characteristic granted to information depending on its truth-value; trust as the ability to produce credible information; authority when the power to influence of an author is accepted, i.e., when readers accept that the source can modify their opinion, knowledge and decisions.

The model describes also two kinds of relationships between the three concepts: an upward link and a downward link. The model is confronted with findings of empirical research on Wikipedia in particular.

URL : https://firstmonday.org/ojs/index.php/fm/article/view/7108/6555

Returning Individual Research Results to Participants

Authors : National Academies of Sciences, Engineering, and Medicine; Health and Medicine Division; Board on Health Sciences Policy; Committee on the Return of Individual-Specific Research Results Generated in Research Laboratories. Editors: Autumn S. Downey, Emily R. Busta, Michelle Mancher, and Jeffrey R. Botkin

When is it appropriate to return individual research results to participants? The immense interest in this question has been fostered by the growing movement toward greater transparency and participant engagement in the research enterprise.

Yet, the risks of returning individual research results—such as results with unknown validity—and the associated burdens on the research enterprise are competing considerations.

Returning Individual Research Results to Participants reviews the current evidence on the benefits, harms, and costs of returning individual research results, while also considering the ethical, social, operational, and regulatory aspects of the practice.

This report includes 12 recommendations directed to various stakeholders—investigators, sponsors, research institutions, institutional review boards (IRBs), regulators, and participants—and are designed to help (1) support decision making regarding the return of results on a study-by-study basis, (2) promote high-quality individual research results, (3) foster participant understanding of individual research results, and (4) revise and harmonize current regulations.

URL : https://www.ncbi.nlm.nih.gov/books/NBK513173/

Reproducible research practices, transparency, and open access data in the biomedical literature, 2015–2017

Authors : Joshua D. Wallach, Kevin W. Boyack, John P. A. Ioannidis

Currently, there is a growing interest in ensuring the transparency and reproducibility of the published scientific literature. According to a previous evaluation of 441 biomedical journals articles published in 2000–2014, the biomedical literature largely lacked transparency in important dimensions.

Here, we surveyed a random sample of 149 biomedical articles published between 2015 and 2017 and determined the proportion reporting sources of public and/or private funding and conflicts of interests, sharing protocols and raw data, and undergoing rigorous independent replication and reproducibility checks.

We also investigated what can be learned about reproducibility and transparency indicators from open access data provided on PubMed. The majority of the 149 studies disclosed some information regarding funding (103, 69.1% [95% confidence interval, 61.0% to 76.3%]) or conflicts of interest (97, 65.1% [56.8% to 72.6%]).

Among the 104 articles with empirical data in which protocols or data sharing would be pertinent, 19 (18.3% [11.6% to 27.3%]) discussed publicly available data; only one (1.0% [0.1% to 6.0%]) included a link to a full study protocol. Among the 97 articles in which replication in studies with different data would be pertinent, there were five replication efforts (5.2% [1.9% to 12.2%]).

Although clinical trial identification numbers and funding details were often provided on PubMed, only two of the articles without a full text article in PubMed Central that discussed publicly available data at the full text level also contained information related to data sharing on PubMed; none had a conflicts of interest statement on PubMed.

Our evaluation suggests that although there have been improvements over the last few years in certain key indicators of reproducibility and transparency, opportunities exist to improve reproducible research practices across the biomedical literature and to make features related to reproducibility more readily visible in PubMed.

URL : Reproducible research practices, transparency, and open access data in the biomedical literature, 2015–2017

DOI : https://doi.org/10.1371/journal.pbio.2006930

“No comment”?: A study of commenting on PLOS articles

Authors : Simon Wakeling, Peter Willett, Claire Creaser, Jenny Fry, Stephen Pinfield, Valerie Spezi, Marc Bonne, Christina Founti, Itzelle Medina Perea

Article commenting functionality allows users to add publically visible comments to an article on a publisher’s website. As well as facilitating forms of post-publication peer review, for publishers of open-access mega-journals (large, broad scope, OA journals that seek to publish all technically or scientifically sound research) comments are also thought to serve as a means for the community to discuss and communicate the significance and novelty of the research, factors which are not assessed during peer review.

In this paper we present the results of an analysis of commenting on articles published by the Public Library of Science (PLOS), publisher of the first and best-known mega-journal PLOS ONE, between 2003 and 2016.

We find that while overall commenting rates are low, and have declined since 2010, there is substantial variation across different PLOS titles. Using a typology of comments developed for this research we also find that only around half of comments engage in an academic discussion of the article, and that these discussions are most likely to focus on the paper’s technical soundness.

Our results suggest that publishers have yet to encourage significant numbers of readers to leave comments, with implications for the effectiveness of commenting as a means of collecting and communicating community perceptions of an article’s importance.

DOI : https://doi.org/10.1177%2F0165551518819965

Negative Effects of “Predatory” Journals on Global Health Research

Authors : Diego A. Forero, Marilyn H. Oermann, Andrea Manca, Franca Deriu, Hugo Mendieta-Zerón, Mehdi Dadkhah, Roshan Bhad, Smita N. Deshpande, Wei Wang, Myriam Patricia Cifuentes

Predatory journals (PJ) exploit the open-access model promising high acceptance rate and fast track publishing without proper peer review. At minimum, PJ are eroding the credibility of the scientific literature in the health sciences as they actually boost the propagation of errors.

In this article, we identify issues with PJ and provide several responses, from international and interdisciplinary perspectives in health sciences.

Authors, particularly researchers with limited previous experience with international publications, need to be careful when considering potential journals for submission, due to the current existence of large numbers of PJ.

Universities around the world, particularly in developing countries, might develop strategies to discourage their researchers from submitting manuscripts to PJ or serving as members of their editorial committees.

URL : Negative Effects of “Predatory” Journals on Global Health Research

DOI : http://doi.org/10.29024/aogh.2389