Gender and international diversity improves equity in peer review

Authors : Dakota Murray, Kyle Siler, Vincent Lariviére, Wei Mun Chan, Andrew M. Collings, Jennifer Raymond, Cassidy R Sugimoto

The robustness of scholarly peer review has been challenged by evidence of disparities in publication outcomes based on author’s gender and nationality. To address this, we examine the peer review outcomes of 23,873 initial submissions and 7,192 full submissions that were submitted to the biosciences journal eLife between 2012 and 2017.

Women and authors from nations outside of North America and Europe were underrepresented both as gatekeepers (editors and peer reviewers) and last authors. We found a homophilic interaction between the demographics of the gatekeepers and authors in determining the outcome of peer review; that is, gatekeepers favor manuscripts from authors of the same gender and from the same country.

The acceptance rate for manuscripts with male last authors was significantly higher than for female last authors, and this gender inequity was greatest when the team of reviewers was all male; mixed-gender gatekeeper teams lead to more equitable peer review outcomes.

Similarly, manuscripts were more likely to be accepted when reviewed by at least one gatekeeper with the same national affiliation as the corresponding author. Our results indicated that homogeneity between author and gatekeeper gender and nationality is associated with the outcomes of scientific peer review.

We conclude with a discussion of mechanisms that could contribute to this effect, directions for future research, and policy implications. Code and anonymized data have been made available at https://github.com/murrayds/elife-analysis

URL : Gender and international diversity improves equity in peer review

DOI : https://doi.org/10.1101/400515

Mesurer la science

Auteurs/Authors : Vincent Larivière, Cassidy R. Sugimoto

L’ensemble de la communauté scientifique réclame depuis plusieurs années des indicateurs fiables permettant de mesurer les répercussions de la recherche. La ferveur inégalée autour de la mesure de l’influence de la recherche, combinée avec les nouveaux modes de diffusion des connaissances à l’ère numérique, a révolu­tionné le domaine de la scientométrie.

Il s’agit là d’une discipline qui comprend toutes les façons dont nous collectons les documents savants et analysons quantitativement leur production ainsi que leurs usages, des citations aux tweets. Les données et les indicateurs ainsi recueillis sont utilisés pour comprendre la science, stimuler la recherche ou distribuer les ressources.

Curieusement, il n’existe aucun ouvrage qui explique les fonde­ments historiques, les concepts et les sources de la scientométrie, ou qui en fournirait une critique éclairée ou même qui formulerait des recommandations pour un usage optimal. D’où l’importance de celui-ci.

À sa façon, chacun est un acteur de la société du savoir et devrait se soucier des outils qui aident à guider son évolution : c’est pourquoi ce livre s’adresse à tous, savants comme profanes.

URL : https://pum.umontreal.ca/catalogue/mesurer-la-science/

Vanishing industries and the rising monopoly of universities in published research

Authors : Vincent Larivière, Benoit Macaluso, Philippe Mongeon, Kyle Siler, Cassidy R. Sugimoto

Anecdotes abound regarding the decline of basic research in industrial and governmental settings, but very little empirical evidence exists about the phenomenon. This article provides a systematic and historical analysis of the contribution of various institutional sectors to knowledge production at the world and country levels across the past four decades.

It highlights a dramatic decline in the diffusion of basic research by industrial and governmental sectors across all countries—with a corresponding increase in the share from universities—as well as an increase of partnerships between universities and other sectors.

Results also shows an increase in the relative share of industries in applied research, as measured through patents. Such divergence in university and industry research activities may hinder industries’ ability to translate basic knowledge into technological innovation, and could lead to a growing misalignment between doctoral training and future job expectations.

Industries and universities must rethink strategies for partnerships and publishing to maximize scientific progress and to ensure the greatest gains for society.

URL : Vanishing industries and the rising monopoly of universities in published research

DOI : https://doi.org/10.1371/journal.pone.0202120

The Journal Impact Factor: A brief history, critique, and discussion of adverse effects

Authors : Vincent Lariviere, Cassidy R. Sugimoto

The Journal Impact Factor (JIF) is, by far, the most discussed bibliometric indicator. Since its introduction over 40 years ago, it has had enormous effects on the scientific ecosystem: transforming the publishing industry, shaping hiring practices and the allocation of resources, and, as a result, reorienting the research activities and dissemination practices of scholars.

Given both the ubiquity and impact of the indicator, the JIF has been widely dissected and debated by scholars of every disciplinary orientation. Drawing on the existing literature as well as on original research, this chapter provides a brief history of the indicator and highlights well-known limitations-such as the asymmetry between the numerator and the denominator, differences across disciplines, the insufficient citation window, and the skewness of the underlying citation distributions.

The inflation of the JIF and the weakening predictive power is discussed, as well as the adverse effects on the behaviors of individual actors and the research enterprise. Alternative journal-based indicators are described and the chapter concludes with a call for responsible application and a commentary on future developments in journal indicators

URL : https://arxiv.org/abs/1801.08992

A systematic identification and analysis of scientists on Twitter

Authors : Qing Ke, Yong-Yeol Ahn, Cassidy R. Sugimoto

Metrics derived from Twitter and other social media—often referred to as altmetrics—are increasingly used to estimate the broader social impacts of scholarship. Such efforts, however, may produce highly misleading results, as the entities that participate in conversations about science on these platforms are largely unknown.

For instance, if altmetric activities are generated mainly by scientists, does it really capture broader social impacts of science? Here we present a systematic approach to identifying and analyzing scientists on Twitter.

Our method can identify scientists across many disciplines, without relying on external bibliographic data, and be easily adapted to identify other stakeholder groups in science.

We investigate the demographics, sharing behaviors, and interconnectivity of the identified scientists.

We find that Twitter has been employed by scholars across the disciplinary spectrum, with an over-representation of social and computer and information scientists; under-representation of mathematical, physical, and life scientists; and a better representation of women compared to scholarly publishing.

Analysis of the sharing of URLs reveals a distinct imprint of scholarly sites, yet only a small fraction of shared URLs are science-related. We find an assortative mixing with respect to disciplines in the networks between scientists, suggesting the maintenance of disciplinary walls in social media.

Our work contributes to the literature both methodologically and conceptually—we provide new methods for disambiguating and identifying particular actors on social media and describing the behaviors of scientists, thus providing foundational information for the construction and use of indicators on the basis of social media metrics.

 URL : A systematic identification and analysis of scientists on Twitter

DOI : https://doi.org/10.1371/journal.pone.0175368

arXiv e-prints and the journal of record: An analysis of roles and relationships

Authors : Vincent Larivière, Cassidy R. Sugimoto, Benoit Macaluso, Staša Milojević, Blaise Cronin, Mike Thelwal

Since its creation in 1991, arXiv has become central to the diffusion of research in a number of fields. Combining data from the entirety of arXiv and the Web of Science (WoS), this paper investigates (a) the proportion of papers across all disciplines that are on arXiv and the proportion of arXiv papers that are in the WoS, (b) elapsed time between arXiv submission and journal publication, and (c) the aging characteristics and scientific impact of arXiv e-prints and their published version.

It shows that the proportion of WoS papers found on arXiv varies across the specialties of physics and mathematics, and that only a few specialties make extensive use of the repository.

Elapsed time between arXiv submission and journal publication has shortened but remains longer in mathematics than in physics. In physics, mathematics, as well as in astronomy and astrophysics, arXiv versions are cited more promptly and decay faster than WoS papers.

The arXiv versions of papers – both published and unpublished – have lower citation rates than published papers, although there is almost no difference in the impact of the arXiv versions of both published and unpublished papers.”

URL : http://arxiv.org/abs/1306.3261