Judging Journals: How Impact Factor and Other Metrics Differ across Disciplines

Authors : Quinn Galbraith, Alexandra Carlile Butterfield, Chase Cardon

Given academia’s frequent use of publication metrics and the inconsistencies in metrics across disciplines, this study examines how various disciplines are treated differently by metric systems. We seek to offer academic librarians, university rank and tenure committees, and other interested individuals guidelines for distinguishing general differences between journal bibliometrics in various disciplines.

This study addresses the following questions: How well represented are different disciplines in the indexing of each metrics system (Eigenfactor, Scopus, Web of Science, Google Scholar)? How does each metrics system treat disciplines differently, and how do these differences compare across metrics systems?

For university libraries and academic librarians, this study may increase understanding of the comparative value of various metrics, which hopefully will facilitate more informed decisions regarding the purchase of journal subscriptions and the evaluation of journals and metrics systems.

This study indicates that different metrics systems prioritize different disciplines, and metrics are not always easily compared across disciplines. Consequently, this study indicates that simple reliance on metrics in publishing or purchasing decisions is often flawed.

URL : Judging Journals: How Impact Factor and Other Metrics Differ across Disciplines

DOI : https://doi.org/10.5860/crl.84.6.888

Measured in a context: making sense of open access book data

Author : Ronald Snijder

Open access (OA) book platforms, such as JSTOR, OAPEN Library or Google Books, have been available for over a decade. Each platform shows usage data, but this results in confusion about how well an individual book is performing overall. Even within one platform, there are considerable usage differences between subjects and languages. Some context is therefore necessary to make sense of OA books usage data.

A possible solution is a new metric – the Transparent Open Access Normalized Index (TOANI) score. It is designed to provide a simple answer to the question of how well an individual open access book or chapter is performing. The transparency is based on clear rules, and by making all of the data used visible.

The data is normalized, using a common scale for the complete collection of an open access book platform and, to keep the level of complexity as low as possible, the score is based on a simple metric.

As a proof of the concept, the usage of over 18,000 open access books and chapters in the OAPEN Library has been analysed, to determine whether each individual title has performed as well as can be expected compared to similar titles.

URL : Measured in a context: making sense of open access book data

DOI : https://doi.org/10.1629/uksg.627

Beyond views, productivity, and citations: measuring geopolitical differences of scientific impact in communication research

Authors : János József Tóth, Gergő Háló, Manuel Goyanes

Scientometric analyses applying critical sociological frameworks have previously shown that high-prestige research output—with regards to both quantity and impact—is typically clustered in a few core countries and world regions, indicating uneven power relations and systematic biases within global academia.

Although citation count is a common formula in these analyses, only a handful of studies investigated altmetrics (impact measures beyond citation-based metrics) in communication science. In this paper, we explore geopolitical biases of impact amongst the most productive scholars in the field of communication from 11 countries and 3 world regions.

Drawing on SCOPUS data, we test three formulas that measure scholarly performance (citations per document; views per document; and citations per view) to investigate how geographical location affects the impact of scholars. Our results indicate a strong US-dominance with regard to citation-based impact, emphasizing a further need for de-Westernization within the field.

Moreover, the analysis of altmetric formulas revealed that research published by Eastern European and Spanish scholars, although accessed similarly or even more often than American or Western European publications, is less cited than those. Country-level comparisons are also discussed.

URL : Beyond views, productivity, and citations: measuring geopolitical differences of scientific impact in communication research

DOI : https://doi.org/10.1007/s11192-023-04801-7

Evaluative altmetrics: is there evidence for its application to research evaluation?

Authors : Wenceslao Arroyo-Machado, Daniel Torres-Salinas

Introduction

Altmetrics have been demonstrated as a promising tool for analyzing scientific communication on social media. Nevertheless, its application for research evaluation remains underdeveloped, despite the advancement of research in the study of diverse scientific interactions.

Methods

This paper develops a method for applying altmetrics in the evaluation of researchers, focusing on a case study of the Environment/Ecology ESI field publications by researchers at the University of Granada. We considered Twitter as a mirror of social attention, news outlets as media, and Wikipedia as educational, exploring mentions from these three sources and the associated actors in their respective media, contextualizing them using various metrics.

Results

Our analysis evaluated different dimensions such as the type of audience, local attention, engagement generated around the mention, and the profile of the actor. Our methodology effectively provided dashboards that gave a comprehensive view of the different instances of social attention at the author level.

Discussion

The use of altmetrics for research evaluation presents significant potential, as shown by our case study. While this is a novel method, our results suggest that altmetrics could provide valuable insights into the social attention that researchers garner. This can be an important tool for research evaluation, expanding our understanding beyond traditional metrics.

URL : Evaluative altmetrics: is there evidence for its application to research evaluation?

DOI : https://doi.org/10.3389/frma.2023.1188131

Gender-Related Differences in the Citation Impact of Scientific Publications and Improving the Authors’ Productivity

Authors : Oleksandr Kuchanskyi, Yurii Andrashko, Andrii Biloshchytskyi, Serik Omirbayev, Aidos Mukhatayev, Svitlana Biloshchytska, Adil Faizullin

The article’s purpose is an analysis of the citation impact of scientific publications by authors of different gender compositions. The page method was chosen to calculate the citation impact of scientific publications, and the obtained results allowed to estimate the impact of the scientific publications based on the number of citations.

The normalized citation impact is calculated according to nine subsets of scientific publications that correspond to patterns of different gender compositions of authors. Also, these estimates were calculated for each country with which the authors of the publications are affiliated.

The Citation database, Network Dataset (Ver. 13), was chosen for the scientometric analysis. The dataset includes more than 5 million scientific publications and 48 million citations. Most of the publications in the dataset are from the STEM field. The results indicate that articles with a predominantly male composition are cited more than articles with a mixed or female composition of authors in this direction.

Analysis of advantages in dynamics indicates that in the last decade, in developed countries, there has been a decrease in the connection between the citation impact of scientific publications and the gender composition of their authors.

However, the obtained results still confirm the presence of gender inequality in science, which may be related to socioeconomic and cultural characteristics, natural homophily, and other factors that contribute to the appearance of gender gaps.

An essential consequence of overcoming these gaps, including in science, is ensuring the rights of people in all their diversity.

URL : Gender-Related Differences in the Citation Impact of Scientific Publications and Improving the Authors’ Productivity

DOI : https://doi.org/10.3390/publications11030037

Citation differences across research funding and access modalities

Authors : Pablo Dorta-González, María Isabel Dorta-González

This research provides insight into the complex relationship between open access, funding, and citation advantage. It presents an analysis of research articles and their citations in the Scopus database across 40 subject categories.

The sample includes 12 categories from Health Sciences, 7 from Life Sciences, 10 from Physical Sciences & Engineering, and 11 from Social Sciences & Humanities. Specifically, the analysis focuses on articles published in 2016 and the citations they received from 2016 to 2020.

Our findings show that open access articles published in hybrid journals receive considerably more citations than those published in gold open access journals. Articles under the hybrid gold modality are cited on average twice as much as those in the gold modality, regardless of funding.

Furthermore, we found that funded articles generally obtain 50 % more citations than unfunded ones within the same publication modality. Open access repositories significantly increase citations, particularly for articles without funding. Thus, articles in open access repositories receive 50 % more citations than paywalled ones.

URL : Citation differences across research funding and access modalities

DOI : https://doi.org/10.1016/j.acalib.2023.102734

Influence of research on open science in the public policy sphere

Authors : Daniela De Filippo, Pablo Sastrón‑Toledo

This paper analyses the scientific activity related to open science in Spain and its influence on public policy from a bibliometric perspective. For this purpose, Spanish centres’ projects and publications on open science from 2010 to 2020 are studied. Subsequently, policy documents using papers related to open science are analysed to study their influence on policymaking.

A total of 142 projects and 1491 publications are analysed, 15% of which are mentioned in policy documents.

The publications cited in policy documents display high proportions of international collaboration, open access publication and publication in first-quartile journals. The findings underline governments’ leading role in the implementation of open science policies and the funding of open science research.

The same government agencies that promote and fund open science research are shown to use that research in their institutional reports, a process known as knowledge flow feedback.

Other non-academic actors are also observed to make use of the knowledge produced by open science research, showing how the open science movement has crossed the boundaries of academia.

URL : Influence of research on open science in the public policy sphere

DOI : https://doi.org/10.1007/s11192-023-04645-1