Archives des catégories : EN

Assessment of potential bias in research grant peer review in Canada

Posté le par

Authors : Robyn Tamblyn, Nadyne Girard, Christina J. Qian, James Hanley

BACKGROUND

Peer review is used to determine what research is funded and published, yet little is known about its effectiveness, and it is suspected that there may be biases. We investigated the variability of peer review and factors influencing ratings of grant applications.

METHODS

We evaluated all grant applications submitted to the Canadian Institutes of Health Research between 2012 and 2014. The contribution of application, principal applicant and reviewer characteristics to overall application score was assessed after adjusting for the applicant’s scientific productivity.

RESULTS

Among 11 624 applications, 66.2% of principal applicants were male and 64.1% were in a basic science domain. We found a significant nonlinear association between scientific productivity and final application score that differed by applicant gender and scientific domain, with higher scores associated with past funding success and h-index and lower scores associated with female applicants and those in the applied sciences.

Significantly lower application scores were also associated with applicants who were older, evaluated by female reviewers only (v. male reviewers only, −0.05 points, 95% confidence interval [CI] −0.08 to −0.02) or reviewers in scientific domains different from the applicant’s (−0.07 points, 95% CI −0.11 to −0.03).

Significantly higher application scores were also associated with reviewer agreement in application score (0.23 points, 95% CI 0.20 to 0.26), the existence of reviewer conflicts (0.09 points, 95% CI 0.07 to 0.11), larger budget requests (0.01 points per $100 000, 95% CI 0.007 to 0.02), and resubmissions (0.15 points, 95% CI 0.14 to 0.17).

In addition, reviewers with high expertise were more likely than those with less expertise to provide higher scores to applicants with higher past success rates (0.18 points, 95% CI 0.08 to 0.28).

INTERPRETATION

There is evidence of bias in peer review of operating grants that is of sufficient magnitude to change application scores from fundable to nonfundable. This should be addressed by training and policy changes in research funding.

URL : https://doi.org/10.1503/cmaj.170901

Library-Mediated Deposit: A Gift to Researchers or a Curse on Open Access? Reflections from the Case of Surrey

Posté le par

Authors : Christine Antiope Daoutis, Maria de Montserrat Rodriguez-Marquez

The University of Surrey was one of the first universities to set up an open access repository. The Library was the natural stakeholder to lead this project. Over the years, the service has been influenced by external and internal factors, and consequently the Library’s role in developing the OA agenda has changed.

Here, we present the development and implementation of a fully mediated open access service at Surrey. The mediated workflow was introduced following an operational review, to ensure higher compliance and engagement from researchers.

The size and responsibilities of the open access team in the Library increased to comply with internal and external policies and to implement the fully mediated workflow. As a result, there has been a growth in deposit rates and overall compliance.

We discuss the benefits and shortcomings of Library mediation; its effects on the relationship between the Library, senior management and researchers, and the increasing necessity for the Library to lead towards a culture of openness beyond policy compliance.

URL : Library-Mediated Deposit: A Gift to Researchers or a Curse on Open Access? Reflections from the Case of Surrey

Alternative location : http://www.mdpi.com/2304-6775/6/2/20

Is Predatory Publishing a Real Threat? Evidence from a Large Database Study

Posté le par

Authors : Marcelo Perlin, Takeyoshi Imasato, Denis Borenstein

Using a database of potential, possible, or probable predatory scholarly open-access journals, the objective of this research is to study the penetration of predatory publications in the Brazilian academic system and the profile of authors in a cross-section empirical study.

Based on a massive amount of publications from Brazilian researchers of all disciplines during the 2000 to 2015 period, we were able to analyze the extent of predatory publications using an econometric modeling.

Descriptive statistics indicate that predatory publications represent a small overall proportion, but grew exponentially in the last 5 years. Departing from prior studies, our analysis shows that experienced researchers with a high number of non-indexed publications and PhD obtained locally are more likely to publish in predatory journals.

Further analysis shows that once a journal regarded as predatory is listed in the local ranking system, the Qualis, it starts to receive more publications than non-predatory ones.

DOI : https://dx.doi.org/10.2139/ssrn.3067958

Are the FAIR Data Principles Fair?

Posté le par

Authors : Alastair Dunning, Madeleine de Smaele, Jasmin Böhmer

This practice paper describes an ongoing research project to test the effectiveness and relevance of the FAIR Data Principles. Simultaneously, it will analyse how easy it is for data archives to adhere to the principles. The research took place from November 2016 to January 2017, and will be underpinned with feedback from the repositories.

The FAIR Data Principles feature 15 facets corresponding to the four letters of FAIR – Findable, Accessible, Interoperable, Reusable. These principles have already gained traction within the research world.

The European Commission has recently expanded its demand for research to produce open data. The relevant guidelines1are explicitly written in the context of the FAIR Data Principles. Given an increasing number of researchers will have exposure to the guidelines, understanding their viability and suggesting where there may be room for modification and adjustment is of vital importance.

This practice paper is connected to a dataset (Dunning et al.,2017) containing the original overview of the sample group statistics and graphs, in an Excel spreadsheet. Over the course of two months, the web-interfaces, help-pages and metadata-records of over 40 data repositories have been examined, to score the individual data repository against the FAIR principles and facets.

The traffic-light rating system enables colour-coding according to compliance and vagueness. The statistical analysis provides overall, categorised, on the principles focussing, and on the facet focussing results.

The analysis includes the statistical and descriptive evaluation, followed by elaborations on Elements of the FAIR Data Principles, the subject specific or repository specific differences, and subsequently what repositories can do to improve their information architecture.

URL : Are the FAIR Data Principles Fair?

DOI: https://doi.org/10.2218/ijdc.v12i2.567

 

Integration of an Active Research Data System with a Data Repository to Streamline the Research Data Lifecyle: Pure-NOMAD Case Study

Posté le par

Authors : Simone Ivan Conte, Federica Fina, Michalis Psalios, Shyam Ryal, Tomas Lebl, Anna Clements

Research funders have introduced requirements that expect researchers to properly manage and publicly share their research data, and expect institutions to put in place services to support researchers in meeting these requirements.

So far the general focus of these services and systems has been on addressing the final stages of the research data lifecycle (archive, share and re-use), rather than stages related to the active phase of the cycle (collect/create and analyse).

As a result, full integration of active data management systems with data repositories is not yet the norm, making the streamlined transition of data from an active to a published and archived status an important challenge.

In this paper we present the integration between an active data management system developed in-house (NOMAD) and Elsevier’s Pure data repository used at our institution, with the aim of offering a simple workflow to facilitate and promote the data deposit process.

The integration results in a new data management and publication workflow that helps researchers to save time, minimize human errors related to manually handling files, and further promote data deposit together with collaboration across the institution.

URL : Integration of an Active Research Data System with a Data Repository to Streamline the Research Data Lifecyle: Pure-NOMAD Case Study

The Journal Impact Factor: A brief history, critique, and discussion of adverse effects

Posté le par

Authors : Vincent Lariviere, Cassidy R. Sugimoto

The Journal Impact Factor (JIF) is, by far, the most discussed bibliometric indicator. Since its introduction over 40 years ago, it has had enormous effects on the scientific ecosystem: transforming the publishing industry, shaping hiring practices and the allocation of resources, and, as a result, reorienting the research activities and dissemination practices of scholars.

Given both the ubiquity and impact of the indicator, the JIF has been widely dissected and debated by scholars of every disciplinary orientation. Drawing on the existing literature as well as on original research, this chapter provides a brief history of the indicator and highlights well-known limitations-such as the asymmetry between the numerator and the denominator, differences across disciplines, the insufficient citation window, and the skewness of the underlying citation distributions.

The inflation of the JIF and the weakening predictive power is discussed, as well as the adverse effects on the behaviors of individual actors and the research enterprise. Alternative journal-based indicators are described and the chapter concludes with a call for responsible application and a commentary on future developments in journal indicators

URL : https://arxiv.org/abs/1801.08992

Supporting Open Access at Kent—New Staff Roles

Posté le par

Authors : Rosalyn Bass, Sarah Slowe

Open Access has been supported at the University of Kent from an early stage with the establishment of the Kent Academic Repository in 2007.

Initially, this work was accommodated within the existing library staff structure, but the pace of change, funder requirements, and a new university plan meant that support for Open Access needed to become explicit.

Therefore, a research support team was established using a matrix working system. This article details this new structure and reflects on the benefits and challenges it brings.

URL : Supporting Open Access at Kent—New Staff Roles

Alternative location : http://www.mdpi.com/2304-6775/6/2/17/htm