Archives des mots-clés : research impact

Social Media Analysis of High-Impact Information and Communication Journals: Adoption, Use, and Content Curation

Posté le par

Authors : Jesús Cascón-Katchadourian, Javier Guallar, Wileidys Artigas

The use of social media to disseminate academic content is increasing, particularly in scientific journals. This study has the following two main objectives: first, exploring the use of social media by high-impact academic journals in two different SJR categories (Library and Information Sciences and Communication), and second, analyzing content curation carried out by the world’s most influential journals in both areas. The research methodology is descriptive with a quantitative approach regarding the items studied.

The study finds that COM journals have a stronger social media presence than LIS journals, and X dominates in both categories and regions as the top social network, with significant influence as the only platform. On the other hand, content curation was found to a high degree in both areas, especially in the LIS area, with 93% vs. 80% in COM. The study highlights that both COM and LIS journals primarily focus on promoting recent articles, with COM diversifying content more than LIS. In terms of the content curation techniques used in both areas, the majority are abstracting and summarizing.

URL : Social Media Analysis of High-Impact Information and Communication Journals: Adoption, Use, and Content Curation

DOI : https://doi.org/10.3390/publications13010005

Improving the reporting of research impact assessments: a systematic review of biomedical funder research impact assessments

Posté le par

Authors : Rachel Abudu, Kathryn Oliver, Annette Boaz

The field of research impact assessment (RIA) has seen remarkable growth over the past three decades. Increasing numbers of RIA frameworks have been developed and applied by research funders and new technologies can capture some research impacts automatically. However, RIAs are too different to draw comparable conclusions about what type of methods, data or processes are best suited to assess research impacts of different kinds, or how funders should most efficiently implement RIAs.

To usher in the next era of RIA and mature the field, future RIA methodologies should become more transparent, standardized and easily implementable. Key to these efforts is an improved understanding of how to practically implement and report on RIA at the funder-level. Our aim is to address this gap through two major contributions.

First, we identify common items across existing best practice guidelines for RIA, creating a preliminary reporting checklist for standardized RIA reporting. Next, we systematically reviewed studies examining funders’ assessment of biomedical grant portfolios to examine how funders reported the results of their RIAs across the checklist, as well as the operational steps funders took to perform their RIA and the variation in how funders implemented the same RIA frameworks.

We compare evidence on current RIA practices with the reporting checklist to identify good practice for RIA reporting, gaps in the evidence base for future research, and recommendations for future effective RIA.

URL : Improving the reporting of research impact assessments: a systematic review of biomedical funder research impact assessments

DOI : https://doi.org/10.1093/reseval/rvae060

Does Open Access Foster Interdisciplinary Citation? Decomposing Open Access Citation Advantage

Posté le par

Authors : Kai Nishikawa, Akiyoshi Murakami

The existence of an open access (OA) citation advantage, that is, whether OA increases citations, has been a topic of interest for many years. Although numerous previous studies have focused on whether OA increases citations, expectations for OA go beyond that. One such expectation is the promotion of knowledge transfer across various fields.

This study aimed to clarify whether OA, especially gold OA, increases interdisciplinary citations in various natural science fields. Specifically, we measured the effect of OA on interdisciplinary and within-discipline citation counts by decomposing an existing metric of the OA citation advantage.

The results revealed that OA increases both interdisciplinary and within-discipline citations in many fields and increases only interdisciplinary citations in chemistry, computer science, and clinical medicine. Among these fields, clinical medicine tends to obtain more interdisciplinary citations without being influenced by specific journals or papers.

The findings indicate that OA fosters knowledge transfer to different fields, which extends our understanding of its effects.

Arxiv : https://arxiv.org/abs/2411.14653

The use of ChatGPT for identifying disruptive papers in science: a first exploration

Posté le par

Authors : Lutz Bornmann, Lingfei Wu, Christoph Ettl

ChatGPT has arrived in quantitative research evaluation. With the exploration in this Letter to the Editor, we would like to widen the spectrum of the possible use of ChatGPT in bibliometrics by applying it to identify disruptive papers.

The identification of disruptive papers using publication and citation counts has become a popular topic in scientometrics. The disadvantage of the quantitative approach is its complexity in the computation. The use of ChatGPT might be an easy to use alternative.

URL : The use of ChatGPT for identifying disruptive papers in science: a first exploration

DOI : https://doi.org/10.1007/s11192-024-05176-z

Open access improves the dissemination of science: insights from Wikipedia

Posté le par

Authors : Puyu Yang, Ahad Shoaib, Robert West, Giovanni Colavizza

Wikipedia is a well-known platform for disseminating knowledge, and scientific sources, such as journal articles, play a critical role in supporting its mission. The open access movement aims to make scientific knowledge openly available, and we might intuitively expect open access to help further Wikipedia’s mission. However, the extent of this relationship remains largely unknown.

To fill this gap, we analyse a large dataset of citations from the English Wikipedia and model the role of open access in Wikipedia’s citation patterns. Our findings reveal that Wikipedia relies on open access articles at a higher overall rate (44.1%) compared to their availability in the Web of Science (23.6%) and OpenAlex (22.6%). Furthermore, both the accessibility (open access status) and academic impact (citation count) significantly increase the probability of an article being cited on Wikipedia.

Specifically, open access articles are extensively and increasingly more cited in Wikipedia, as they show an approximately 64.7% higher likelihood of being cited in Wikipedia when compared to paywalled articles, after controlling for confounding factors. This open access citation effect is particularly strong for articles with high citation counts or published in recent years.

Our findings highlight the pivotal role of open access in facilitating the dissemination of scientific knowledge, thereby increasing the likelihood of open access articles reaching a more diverse audience through platforms such as Wikipedia. Simultaneously, open access articles contribute to the reliability of Wikipedia as a source by affording editors timely access to novel results.

URL : Open access improves the dissemination of science: insights from Wikipedia

DOI : https://doi.org/10.1007/s11192-024-05163-4

Economic valuation of open research data: A conceptual framework and methodological approach

Posté le par

Authors : Zhifang Tu, Jiashu Shen

The economic significance of open research data is widely acknowledged, yet its quantification remains challenging. This paper presents an effective valuation instrument to help stakeholders understand and evaluate the economic benefits of open research data. By conducting a scoping review and prioritizing user engagement, this study introduces a comprehensive conceptual framework for the economic valuation of open research data. The valuation is based on economic value and willingness to pay, employing the Contingent Valuation Method (CVM).

It incorporates per-use models (view, download, and request) and periodic subscription models (monthly and yearly). An empirical survey was conducted at the National Basic Science Data Center (NBSDC) in China to verify this framework. Both pricing models, comprising five distinct tactics, was supported by surveyed users. Measuring economic value by views and by year was preferred, while willingness to pay by downloads and by year was considered more reasonable.

Overall, the most applicable valuation approach is on a yearly basis. Through this case study at NBSDC, specific pricing tactics were identified, and the total economic value and users’ willingness to pay were assessed. This study is arguably the first to establish a conceptual framework with pricing tactics from a user perspective. This methodological approach for economic valuation of open research data provides evidence and tools for future research, policy formulation, and resource allocation in the context of open science and innovation.

URL : Economic valuation of open research data: A conceptual framework and methodological approach

DOI : https://doi.org/10.1093/reseval/rvae033

The role of non-scientific factors vis-à-vis the quality of publications in determining their scholarly impact

Posté le par

Authors : Giovanni Abramo, Ciriaco Andrea D’Angelo, Leonardo Grilli

In the evaluation of scientific publications’ impact, the interplay between intrinsic quality and non-scientific factors remains a subject of debate. While peer review traditionally assesses quality, bibliometric techniques gauge scholarly impact.

This study investigates the role of non-scientific attributes alongside quality scores from peer review in determining scholarly impact. Leveraging data from the first Italian Research Assessment Exercise (VTR 2001–2003) and Web of Science citations, we analyse the relationship between quality scores, non-scientific factors, and publication short- and long-term impact.

Our findings shed light on the significance of non-scientific elements overlooked in peer review, offering policymakers and research management insights in choosing evaluation methodologies. Sections delve into the debate, identify non-scientific influences, detail methodologies, present results, and discuss implications.

URL : The role of non-scientific factors vis-à-vis the quality of publications in determining their scholarly impact

DOI : https://doi.org/10.1007/s11192-024-05106-z