The role of non-scientific factors vis-a-vis the quality of publications in determining their scholarly impact

Authors : Giovanni Abramo, Ciriaco Andrea D’Angelo, Leonardo Grilli

In the evaluation of scientific publications’ impact, the interplay between intrinsic quality and non-scientific factors remains a subject of debate. While peer review traditionally assesses quality, bibliometric techniques gauge scholarly impact. This study investigates the role of non-scientific attributes alongside quality scores from peer review in determining scholarly impact.

Leveraging data from the first Italian Research Assessment Exercise (VTR 2001-2003) and Web of Science citations, we analyse the relationship between quality scores, non-scientific factors, and publication short- and long-term impact.

Our findings shed light on the significance of non-scientific elements overlooked in peer review, offering policymakers and research management insights in choosing evaluation methodologies. Sections delve into the debate, identify non-scientific influences, detail methodologies, present results, and discuss implications.

Arxiv : https://arxiv.org/abs/2404.05345

Research collaboration and productivity: is there correlation?

Authors : Giovanni Abramo, Ciriaco Andrea D’Angelo, Flavia Di Costa

The incidence of extramural collaboration in academic research activities is increasing as a result of various factors. These factors include policy measures aimed at fostering partnership and networking among the various components of the research system, policies which are in turn justified by the idea that knowledge sharing could increase the effectiveness of the system.

Over the last two decades, the scientific community has also stepped up activities to assess the actual impact of collaboration intensity on the performance of research systems.

This study draws on a number of empirical analyses, with the intention of measuring the effects of extramural collaboration on research performance and, indirectly, verifying the legitimacy of policies that support this type of collaboration.

The analysis focuses on the Italian academic research system. The aim of the work is to assess the level of correlation, at institutional level, between scientific productivity and collaboration intensity as a whole, both internationally and with private organizations.

This will be carried out using a bibliometric type of approach, which equates collaboration with the co-authorship of scientific publications.

URL : https://arxiv.org/abs/1812.07847

Do all citations value the same? Valuing citations by the value of the citing items

Authors : Cristiano Giuffrida, Giovanni Abramo, Ciriaco Andrea D’Angelo

Bibliometricians have long recurred to citation counts to measure the impact of publications on the advancement of science. However, since the earliest days of the field, some scholars have questioned whether all citations should value the same, and have gone on to weight them by a variety of factors.

However sophisticated the operationalization of the measures, the methodologies used in weighting citations still present limits in their underlying assumptions. This work takes an alternate approach to resolving the underlying problem: the proposal is to value citations by the impact of the citing articles.

As well as conceptualizing a new indicator of impact, the work illustrates its application to the 2004-2012 Italian scientific production indexed in the WoS.

The new indicator appears highly correlated to traditional field normalized citations, however the shifts observed between the two measures are frequent and the number of outliers not at all negligible. Moreover, the new indicator seems to show greater “sensitivity” when used in identification of the top-cited papers.

URL : https://arxiv.org/abs/1809.06088