Étiquette : metrics

Catégories
EN

Constructing a (Meaningful) Scholarly Publishing Narrative: Bootstrapping Green Open Access Submissions for Open Access Week

Authors : Tim Ribaric, Erin Moorhead

Introduction

Green Open Access requires authors to submit appropriate versions of journal articles to a repository; the challenge for researchers is to remember to perform this task after the primary work of the publication process is done. This case study at Brock University Library shows how it is possible to bootstrap these types of submissions by crafting meaningful publishing narratives through a collection of clever Open Access Week initiatives.

Literature Review

The literature review assesses the existing scholarly material on Open Access outreach efforts and touches on Open Access in the Canadian context. It explores the dynamics of self-archiving with Green Open Access and the use of institutional repositories. The existing literature on wielding open data as a tool to serve libraries is discussed, and the potential of these tools to support librarian advocacy work is examined.

Description of Program and Benefits

By using a collection of Open Access scholarly publishing web services and APIs, two outreach campaigns were conducted by Brock University Library during Open Access Week 2025. The first was a State of Scholarly Publishing Report that showed broad trends in output from the organization, and the second was a Green Open Access Call to Action that prompted researchers to submit recently published eligible papers to the repository.

Conclusion

There was enough enthusiasm generated from these two projects that the Brock University Library will continue to run these campaigns in subsequent Open Access Weeks. The publishing report was engaging and was included in many different venues on campus, while the Green Open Access Call to Action resulted in submissions from 37% of contacted authors.

URL : Constructing a (Meaningful) Scholarly Publishing Narrative: Bootstrapping Green Open Access Submissions for Open Access Week

DOI : https://doi.org/10.31274/jlsc.21406

Catégories
EN

Towards a more informed and balanced use of scientific performance metrics

Authors :  Jaap J A Denissen, Klaas Sijtsma, Wil M P van der Aalst

The goal of scientific assessment is to predict which individuals can make optimal use of limited resources within a specific context to make optimal allocation decisions. In academic contexts that pertain to individual-level allocations, this is most relevant for decisions on whom to hire for academic positions, nominate for awards, or whose research projects to fund.

The current perspective paper draws upon insights from decades of psychometric research and more recent research on scientific performance to derive a set of five psychometric criteria that should be met for optimal assessment procedures in academia. Although data-driven decision making has gained popularity in most domains, there is increasing resistance against using quantitative measurements in scientific assessment.

Recently, several stakeholders have proposed to jettison such measurements and focus instead on qualitative indicators or narratives. We argue that both quantitative and qualitative assessment do not always meet our five criteria, but solely relying on qualitative indicators appears to be a suboptimal strategy.

We argue instead that there are smarter ways to use quantitative indicators so that they become more reliable, predictive, and ultimately also more efficient and equitable. We conclude with a set of recommendations for scientific quality assessment that is based on the most recent psychometric and scientific insights. In an appendix, we apply these recommendations to a Dutch case study of how researcher information is considered in the application procedure for a prestigious individual grant.

URL : Towards a more informed and balanced use of scientific performance metrics

DOI : https://doi.org/10.1093/reseval/rvag023

Catégories
EN

Evaluating Open Access Advantages for Citations and Altmetrics (2011-21): A Dynamic and Evolving Relationship

Author : Mike Taylor

Differences between the impacts of Open Access (OA) and non-OA research have been observed over a range of citation and altmetric indicators, usually finding an Open Access Advantage (OAA). However, science-wide analyses covering multiple years, indicators and disciplines are lacking. Using citations and six altmetrics for 33.3M articles published 2011-21, we compare OA and non-OA papers.

The results show that there is no universal OAA across all disciplines or impact indicators: the OAA for citations tends to be lower for recent papers, whereas the OAAs for news, blogs and Twitter are consistent across years and unrelated to volume of OA publications. Wikipedia OAAs are consistently pronounced for all subjects except Humanities (HU) and Social Sciences. Patent OAAs for are strongest for Medical & Health Sciences (MHS) and Life Sciences (LS).

Uniquely, the OAAs for Policy citations is stronger for recently published research. These results support different hypotheses for different subjects and indicators. The evidence is consistent with OA accelerating research impact in MHS, LS and HU; increased visibility/discoverability being a factor in promoting the socio-economic impact; and that OA is a factor in growing online engagement with research. OAAs are therefore complex, dynamic, multi-factorial and require considerable analysis to understand.

URL : Evaluating Open Access Advantages for Citations and Altmetrics (2011-21): A Dynamic and Evolving Relationship

DOI : Serendipity and Scientific Styles: An Ordinary

Catégories
EN

Altmetrics in the evaluation of scholarly impact: a systematic and critical literature review

Authors : Paloma González, Martha Fors, Ariel Torres

Altmetrics have emerged as a complementary tool to traditional citation-based metrics in the assessment of scholarly impact. Unlike traditional metrics that primarily capture academic citations over long periods, altmetrics reflect immediate online attention across platforms such as Twitter, blogs, news outlets, and Mendeley.

This article critically examines whether altmetrics can serve as a substitute for traditional metrics by exploring their strengths, limitations, disciplinary variations, and correlation with conventional indicators.

Through a review of recent empirical studies and theoretical debates, the article argues that while altmetrics offer valuable insights into social impact and engagement, they are not yet mature or standardized enough to fully replace traditional metrics. Instead, a hybrid model that integrates both systems may offer a more holistic and inclusive measure of research influence.

URL : Altmetrics in the evaluation of scholarly impact: a systematic and critical literature review

DOI : https://doi.org/10.3389/frma.2025.1693304

Catégories
EN

Assessing the Societal Impact of Academic Research With Artificial Intelligence (AI): A Scoping Review of Business School Scholarship as a ‘Force for Good’

Authors : David Steingard, Kathleen Rodenburg

This study addresses critical questions about how current evaluative frameworks for academic research can effectively translate scholarly findings into practical applications and policies to tackle societal ‘grand challenges’.

This scoping review analysis was conducted using bibliometric methods and AI tools. Articles were drawn from a wide range of disciplines, with particular emphasis on the business and management fields, focusing on the burgeoning scholarship area of ‘business as a force for good’.

The novel integration of generative AI research approaches underscores the transformative potential of AI-human collaboration in academic research. Metadata from 4051 articles were examined in the scoping review, with only 370 articles (9.1%) explicitly identified as relevant to societal impact.

This finding reveals a substantial and concerning gap in research addressing the urgent social and environmental issues of our time. To address this gap, the study identifies six meta-themes related to enhancing the societal impact of research: business applications; faculty publication pressure; societal impact focus; sustainable development; university and scholarly rankings; and reference to responsible research frameworks.

Key findings highlight critical misalignments between research outputs and the United Nations Sustainable Development Goals (SDGs) and a lack of practical business applications of research insights.

The results emphasise the urgent need for academic institutions to expand evaluation criteria beyond traditional metrics to prioritise real-world impacts. Recommendations include developing holistic evaluation frameworks and incentivising research that addresses pressing societal challenges—shifting academia from a ‘scholar-to-scholar’ to a ‘scholar-to-society’ paradigm.

The implications of this shift are applied to business-related scholarship and its potential to inspire meaningful societal impact through business practice.

URL : Assessing the Societal Impact of Academic Research With Artificial Intelligence (AI): A Scoping Review of Business School Scholarship as a ‘Force for Good’

DOI : https://doi.org/10.1002/leap.2010

Catégories
EN

Does open access foster interdisciplinary citations? Decomposing open access citation advantage

Authors : Kai Nishikawa, Akiyoshi Murakami

The existence of an open access (OA) citation advantage—that is, whether OA increases citations—has been a topic of interest for many years. Although numerous studies have focused on whether OA increases citations, expectations for OA go beyond that. One such expectation is the promotion of knowledge transfer across various fields.

This study aimed to clarify what effects OA, particularly gold OA, has on knowledge transfer across fields. Specifically, we measure the effect of OA on interdisciplinary and within-discipline citation counts by decomposing an existing OA citation advantage metric.

OA increased both interdisciplinary and within-discipline citations in many fields studied, and only interdisciplinary citations in chemistry, computer science, and clinical medicine. In these three fields, clinical medicine showed a tendency toward interdisciplinary citations, independent of journal or paper.

These findings suggest that OA fosters knowledge transfer across disciplines.

URL : Does open access foster interdisciplinary citations? Decomposing open access citation advantage

DOI : https://doi.org/10.1007/s11192-025-05297-z

Catégories
EN

Does Open Access Foster Interdisciplinary Citation? Decomposing Open Access Citation Advantage

Authors : Kai Nishikawa, Akiyoshi Murakami

The existence of an open access (OA) citation advantage, that is, whether OA increases citations, has been a topic of interest for many years. Although numerous previous studies have focused on whether OA increases citations, expectations for OA go beyond that. One such expectation is the promotion of knowledge transfer across various fields.

This study aimed to clarify whether OA, especially gold OA, increases interdisciplinary citations in various natural science fields. Specifically, we measured the effect of OA on interdisciplinary and within-discipline citation counts by decomposing an existing metric of the OA citation advantage.

The results revealed that OA increases both interdisciplinary and within-discipline citations in many fields and increases only interdisciplinary citations in chemistry, computer science, and clinical medicine. Among these fields, clinical medicine tends to obtain more interdisciplinary citations without being influenced by specific journals or papers.

The findings indicate that OA fosters knowledge transfer to different fields, which extends our understanding of its effects.

Arxiv : https://arxiv.org/abs/2411.14653