Archives des mots-clés : research impact

How do scientific papers with different levels of journals spread online? Exploring the temporal dynamics in the diffusion processes

Posté le par

Authors : Renmeng Cao, Xiaoke Xu, Yunxue Cui, Zhizhao Fang, Xianwen Wang

Social media has become an important channel for publicizing academic research, which provides an opportunity for each scientific paper to become a hit.

Employing a dataset of about 10 million tweets of 584,264 scientific papers from 2012 to 2018, this study investigates the differential diffusion of elite and non-elite journal papers (divided by Average journal impact factor percentile).

We find that non-elite journal papers are diffused deeper and farther than elite journal papers, showing a diffusion trend with multiple rounds, sparse, short-duration and small-scale bursts.

In contrast, the bursts of elite journals are characterized by a small number of persistent, dense and large-scale bursts. We also discover that elite journal papers are more inclined to broadcast diffusion while non-elite journal papers prefer viral diffusion.

Elite journal papers are generally disseminated to many loosely connected communities, while non-elite journal papers are diffused to several densely connected communities.

URL : https://arxiv.org/abs/2112.03506

The production of scientific and societal value in research evaluation: a review of societal impact assessment methods

Posté le par

Authors : Jorrit P Smit, Laurens K Hessels

Over the past two decades, several methods have been developed to evaluate the societal impact of research. Compared to the practical development of the field, the conceptual development is relatively weak.

This review article contributes to the latter by elucidating the theoretical aspects of the dominant methods for evaluating societal impact of research, in particular, their presuppositions about the relationship between scientific and societal value of research. We analyse 10 approaches to the assessment of the societal impact of research from a constructivist perspective.

The methods represent different understandings of knowledge exchange, which can be understood in terms of linear, cyclical, and co-production models. In addition, the evaluation methods use a variety of concepts for the societal value of research, which suggest different relationships with scientific value.

While some methods rely on a clear and explicit distinction between the two types of value, other methods, in particular Evaluative Inquiry, ASIRPA, Contribution Mapping, Public Value Mapping, and SIAMPI, consider the mechanisms for producing societal value integral to the research process.

We conclude that evaluation methods must balance between demarcating societal value as a separate performance indicator for practical purposes and doing justice to the (constructivist) science studies’ findings about the integration of scientific and societal value of research.

Our analytic comparison of assessment methods can assist research evaluators in the conscious and responsible selection of an approach that fits with the object under evaluation. As evaluation actively shapes knowledge production, it is important not to use oversimplified concepts of societal value.

URL : The production of scientific and societal value in research evaluation: a review of societal impact assessment methods

DOI : https://doi.org/10.1093/reseval/rvab002

Impact and visibility of Norwegian, Finnish and Spanish journals in the fields of humanities

Posté le par

Authors : Elías Sanz-Casado, Daniela De Filippo, Rafael Aleixandre Benavent, Vidar Røeggen, Janne Pölönen

This article analyses the impact and visibility of scholarly journals in the humanities that are publishing in the national languages in Finland, Norway and Spain. Three types of publishers are considered: commercial publishers, scholarly society as publisher, and research organizations as publishers.

Indicators of visibility and impact were obtained from Web of Science, SCOPUS, Google Metrics, Scimago Journal Rank and Journal Citation Report.

The findings compiled show that in Spain the categories “History and Archaeology” and “Language and Literature” account for almost 70% of the journals analysed, while the other countries offer a more homogeneous distribution.

In Finland, the scholarly society publisher is predominant, in Spain, research organization as publishers, mostly universities, have a greater weighting, while in Norway, the commercial publishers take centre stage.

The results show that journals from Finland and Norway will have reduced possibilities in terms of impact and visibility, since the vernacular language appeals to a smaller readership. Conversely, the Spanish journals are more attractive for indexing in commercial databases. Distribution in open access ranges from 64 to 70% in Norwegian and Finish journals, and to 91% in Spanish journals.

The existence of DOI range from 31 to 41% in Nordic journals to 60% in Spanish journals and has a more widespread bearing on the citations received in all three countries (journals with DOI and open access are cited more frequently).

URL : Impact and visibility of Norwegian, Finnish and Spanish journals in the fields of humanities

DOI : https://doi.org/10.1007/s11192-021-04169-6

The value of research funding for knowledge creation and dissemination: A study of SNSF Research Grants

Posté le par

Authors : Rachel Heyard, Hanna Hottenrott

This study investigates the effect of competitive project funding on researchers’ publication outputs. Using detailed information on applicants at the Swiss National Science Foundation and their proposal evaluations, we employ a case-control design that accounts for individual heterogeneity of researchers and selection into treatment (e.g. funding).

We estimate the impact of the grant award on a set of output indicators measuring the creation of new research results (the number of peer-reviewed articles), its relevance (number of citations and relative citation ratios), as well as its accessibility and dissemination as measured by the publication of preprints and by altmetrics.

The results show that the funding program facilitates the publication and dissemination of additional research amounting to about one additional article in each of the three years following the funding.

The higher citation metrics and altmetrics by funded researchers suggest that impact goes beyond quantity and that funding fosters dissemination and quality.

URL : The value of research funding for knowledge creation and dissemination: A study of SNSF Research Grants

DOI : https://doi.org/10.1057/s41599-021-00891-x

Association between productivity and journal impact across disciplines and career age

Posté le par

Authors : Andre S. Sunahara, Matjaz Perc, Haroldo V. Ribeiro

The association between productivity and impact of scientific production is a long-standing debate in science that remains controversial and poorly understood. Here we present a large-scale analysis of the association between yearly publication numbers and average journal-impact metrics for the Brazilian scientific elite.

We find this association to be discipline-specific, career-age dependent, and similar among researchers with outlier and non-outlier performance. Outlier researchers either outperform in productivity or journal prestige, but they rarely do so in both categories.

Non-outliers also follow this trend and display negative correlations between productivity and journal prestige but with discipline-dependent intensity. Our research indicates that academics are averse to simultaneous changes in their productivity and journal-prestige levels over consecutive career years.

We also find that career patterns concerning productivity and journal prestige are discipline-specific, having in common a raise of productivity with career age for most disciplines and a higher chance of outperforming in journal impact during early career stages.

URL : https://arxiv.org/abs/2108.02920

Who are the users of national open access journals? The case of the Finnish Journal.fi platform

Posté le par

Authors : Janne Pölönen, Sami Syrjämäki, Antti-Jussi Nygård, Björn Hammarfelt

In this paper we study the diversity of users of open access articles on the Finnish Journal.fi platform. This platform hosts around hundred open access journals from Finland publishing in different fields and mainly Finnish and English languages.

The study is based on an online survey, conducted on 48 journals during Spring 2020, in which visitors were asked to indicate their background and allow their location and download behaviour be tracked. Among 668 survey participants, the two largest groups were students (40%) and researchers (36%), followed by private citizens (8%), other experts (7%) and teachers (5%).

Other identified user categories include journalists, civil servants, entrepreneurs and politicians. While new publications attract a considerable share of the views, there is still a relatively large interest, especially among students, in older materials.

Our findings indicate that Finnish language publications are particularly important for reaching students, citizens, experts and politicians. Thus, open access to publications in national languages is vital for the local relevance and outreach of research.

URL : Who are the users of national open access journals? The case of the Finnish Journal.fi platform

DOI : https://doi.org/10.1002/leap.1405

Measuring academic entities’ impact by content-based citation analysis in a heterogeneous academic network

Posté le par

Authors : Fang Zhang, Shengli Wu

Evaluating the impact of papers, researchers and venues objectively is of great significance to academia and beyond. This may help researchers, research organizations, and government agencies in various ways, such as helping researchers find valuable papers and authoritative venues and helping research organizations identify good researchers.

A few studies find that rather than treating citations equally, differentiating them is a promising way for impact evaluation of academic entities. However, most of those methods are metadata-based only and do not consider contents of cited and citing papers; while a few content-based methods are not sophisticated, and further improvement is possible.

In this paper, we study the citation relationships between entities by content-based approaches. Especially, an ensemble learning method is used to classify citations into different strength types, and a word-embedding based method is used to estimate topical similarity of the citing and cited papers.

A heterogeneous network is constructed with the weighted citation links and several other features. Based on the heterogeneous network that consists of three types of entities, we apply an iterative PageRank-like method to rank the impact of papers, authors and venues at the same time through mutual reinforcement.

Experiments are conducted on an ACL dataset, and the results demonstrate that our method greatly outperforms state-of-the art competitors in improving ranking effectiveness of papers, authors and venues, as well as in being robust against malicious manipulation of citations.

URL : Measuring academic entities’ impact by content‑based citation analysis in a heterogeneous academic network

DOI : https://doi.org/10.1007/s11192-021-04063-1