EN

ODDPub – a Text-Mining Algorithm to Detect Data Sharing in Biomedical Publications

Posted on by Hans Dillaerts

Authors: Nico Riedel, Miriam Kip, Evgeny Bobro

Open research data are increasingly recognized as a quality indicator and an important resource to increase transparency, robustness and collaboration in science. However, no standardized way of reporting Open Data in publications exists, making it difficult to find shared datasets and assess the prevalence of Open Data in an automated fashion.

We developed ODDPub (Open Data Detection in Publications), a text-mining algorithm that screens biomedical publications and detects cases of Open Data. Using English-language original research publications from a single biomedical research institution (n = 8689) and randomly selected from PubMed (n = 1500) we iteratively developed a set of derived keyword categories.

ODDPub can detect data sharing through field-specific repositories, general-purpose repositories or the supplement. Additionally, it can detect shared analysis code (Open Code).

To validate ODDPub, we manually screened 792 publications randomly selected from PubMed. On this validation dataset, our algorithm detected Open Data publications with a sensitivity of 0.73 and specificity of 0.97.

Open Data was detected for 11.5% (n = 91) of publications. Open Code was detected for 1.4% (n = 11) of publications with a sensitivity of 0.73 and specificity of 1.00. We compared our results to the linked datasets found in the databases PubMed and Web of Science.

Our algorithm can automatically screen large numbers of publications for Open Data. It can thus be used to assess Open Data sharing rates on the level of subject areas, journals, or institutions. It can also identify individual Open Data publications in a larger publication corpus. ODDPub is published as an R package on GitHub.

URL : ODDPub – a Text-Mining Algorithm to Detect Data Sharing in Biomedical Publications

DOI : http://doi.org/10.5334/dsj-2020-042

EN

Two years into the Brazilian Reproducibility Initiative: reflections on conducting a large-scale replication of Brazilian biomedical science

Posted on by Hans Dillaerts

Author : Kleber Neves, Clarissa FD Carneiro, Ana Paula Wasilewska-Sampaio, Mariana Abreu, Bruna Valério-Gomes, Pedro B Tan, Olavo B Amaral

Scientists have increasingly recognised that low methodological and analytical rigour combined with publish-or-perish incentives can make the published scientific literature unreliable.

As a response to this, large-scale systematic replications of the literature have emerged as a way to assess the problem empirically. The Brazilian Reproducibility Initiative is one such effort, aimed at estimating the reproducibility of Brazilian biomedical research.

Its goal is to perform multicentre replications of a quasi-random sample of at least 60 experiments from Brazilian articles published over a 20-year period, using a set of common laboratory methods.

In this article, we describe the challenges of managing a multicentre project with collaborating teams across the country, as well as its successes and failures over the first two years.

We end with a brief discussion of the Initiative’s current status and its possible future contributions after the project is concluded in 2021.

URL : Two years into the Brazilian Reproducibility Initiative: reflections on conducting a large-scale replication of Brazilian biomedical science

DOI : https://doi.org/10.1590/0074-02760200328

EN

The Enlightenment of Peer Review : How Academic Librarians Can Utilize Open Peer Review Methods to Advance Information Literacy

Posted on by Hans Dillaerts

Author : Sandra Moore

In today’s world of digital scholarly publishing, it is increasingly clear that movements such as open access (OA), Open Science, and open peer review (OPR) are increasingly impactful and gaining momentum.

The shift towards openness in the academy reveals a transformation of traditional structures that compose scholarly communication as well as changing attitudes about the nature of authority and access within these systems.

These new directions in the scholarly information landscape have created a need for academic librarians to realign roles and respond in ways that build resiliency in an era of rapid change.

Recognizing that many core elements of scholarly communication are powerful tools for teaching students about information literacy can lead to transformative instructional strategies.

This paper explores how academic librarians can leverage the innovative traits of OPR to advance information literacy through experiential student learning opportunities grounded in the ACRL (2016) Framework for Information Literacy for Higher Education.

URL : The Enlightenment of Peer Review : How Academic Librarians Can Utilize Open Peer Review Methods to Advance Information Literacy

DOI : https://doi.org/10.5206/elip.v3i1.8618

FR

L’effet SIGAPS : La recherche médicale française sous l’emprise de l’évaluation comptable

Posted on by Hans Dillaerts

Auteurs/Authors : Yves Gingras, Mahdi Khelfaoui

Cette recherche a pour but de mettre en évidence les effets pervers générés par l’introduction du système SIGAPS (Système d’interrogation, de gestion, et d’analyse des publications scientifiques) sur la production scientifique française en médecine et en sciences biomédicales.

Cet outil biblio-métrique de gestion et de financement de la recherche présente un exemple emblématique des dé-rives que peuvent générer les méthodes d’évaluation de la recherche reposant sur des critères pu-rement comptables.

Dans cette note, nous présentons d’abord le fonctionnement de SIGAPS, pour ensuite expliquer précisément en quoi les méthodes de calcul des « points SIGAPS », fondés sur les facteurs d’impact des revues et l’ordre des noms des co-auteurs, posent de nombreux problèmes.

Nous identifions notamment les effets du système SIGAPS sur les dynamiques de publications, les choix des lieux de publications, la langue de publication et les critères de recrutement et de promotion des chercheurs.

Finalement, nous montrons que l’utilisation du système SIGAPS ne répond pas bien à tous les critères de ce que l’on pourrait appeler une « éthique de l’évaluation » qui devrait respecter certaines règles, comme la transparence, l’équité et la validité des indicateurs.

URL : https://cirst2.openum.ca/files/sites/179/2020/10/Note_2020-05vf.pdf

EN

Gender gaps in research productivity and recognition among elite scientists in the U.S., Canada, and South Africa

Posted on by Hans Dillaerts

Authors : Creso Sá, Summer Cowley, Magdalena Martinez, Nadiia Kachynska, Emma Sabzalieva

This study builds upon the literature documenting gender disparities in science by investigating research productivity and recognition among elite scientists in three countries. This analysis departs from both the general comparison of researchers across organizational settings and academic appointments on one hand, and the definition of “elite” by the research outcome variables on the other, which are common in previous studies.

Instead, this paper’s approach considers the stratification of scientific careers by carefully constructing matched samples of men and women holding research chairs in Canada, the United States and South Africa, along with a control group of departmental peers.

The analysis is based on a unique, hand-curated dataset including 943 researchers, which allows for a systematic comparison of successful scientists vetted through similar selection mechanisms.

Our results show that even among elite scientists a pattern of stratified productivity and recognition by gender remains, with more prominent gaps in recognition. Our results point to the need for gender equity initiatives in science policy to critically examine assessment criteria and evaluation mechanisms to emphasize multiple expressions of research excellence.

URL : Gender gaps in research productivity and recognition among elite scientists in the U.S., Canada, and South Africa

DOI : https://doi.org/10.1371/journal.pone.0240903

EN

Scientific data management in the federal government: A case study of NOAA and responsibility for preserving digital data

Posted on by Hans Dillaerts

Authors : Adam Kriesberg, Jacob Kowall

In this paper, we examine the ways in which the evolution of federal and agency‐specific data management policies has affected and continues to affect the long‐term preservation of digital scientific data produced by the United States government.

After reviewing the existing literature on the role of archival theory and practice in the preservation of scientific data, we present the case of the National Oceanic and Atmospheric Administration (NOAA) to analyze how data management activities at this agency are shaped by legislative mandates as well as both government‐wide and agency‐specific information‐management policies.

Through the connected network of law, federal policy, agency policy, and the records schedules which govern recordkeeping practice in the federal government, we propose a number of further questions on how government agencies can effectively provide for the management of scientific data as federal records.

DOI : https://doi.org/10.1002/pra2.266

EN

Data sharing policies of journals in life, health, and physical sciences indexed in Journal Citation Reports

Posted on by Hans Dillaerts

Authors : Jihyun Kim, Soon Kim, Hye-Min Cho, Jae Hwa Chang, Soo Young Kim

Background

Many scholarly journals have established their own data-related policies, which specify their enforcement of data sharing, the types of data to be submitted, and their procedures for making data available.

However, except for the journal impact factor and the subject area, the factors associated with the overall strength of the data sharing policies of scholarly journals remain unknown.

This study examines how factors, including impact factor, subject area, type of journal publisher, and geographical location of the publisher are related to the strength of the data sharing policy.

Methods

From each of the 178 categories of the Web of Science’s 2017 edition of Journal Citation Reports, the top journals in each quartile (Q1, Q2, Q3, and Q4) were selected in December 2018. Of the resulting 709 journals (5%), 700 in the fields of life, health, and physical sciences were selected for analysis.

Four of the authors independently reviewed the results of the journal website searches, categorized the journals’ data sharing policies, and extracted the characteristics of individual journals.

Univariable multinomial logistic regression analyses were initially conducted to determine whether there was a relationship between each factor and the strength of the data sharing policy.

Based on the univariable analyses, a multivariable model was performed to further investigate the factors related to the presence and/or strength of the policy.

Results

Of the 700 journals, 308 (44.0%) had no data sharing policy, 125 (17.9%) had a weak policy, and 267 (38.1%) had a strong policy (expecting or mandating data sharing). The impact factor quartile was positively associated with the strength of the data sharing policies.

Physical science journals were less likely to have a strong policy relative to a weak policy than Life science journals (relative risk ratio [RRR], 0.36; 95% CI [0.17–0.78]). Life science journals had a greater probability of having a weak policy relative to no policy than health science journals (RRR, 2.73; 95% CI [1.05–7.14]).

Commercial publishers were more likely to have a weak policy relative to no policy than non-commercial publishers (RRR, 7.87; 95% CI, [3.98–15.57]). Journals by publishers in Europe, including the majority of those located in the United Kingdom and the Netherlands, were more likely to have a strong data sharing policy than a weak policy (RRR, 2.99; 95% CI [1.85–4.81]).

Conclusions

These findings may account for the increase in commercial publishers’ engagement in data sharing and indicate that European national initiatives that encourage and mandate data sharing may influence the presence of a strong policy in the associated journals.

Future research needs to explore the factors associated with varied degrees in the strength of a data sharing policy as well as more diverse characteristics of journals related to the policy strength.

URL : Data sharing policies of journals in life, health, and physical sciences indexed in Journal Citation Reports

DOI : https://doi.org/10.7717/peerj.9924