Étiquette : Bibliometrics

Catégories
EN

Reshaping How Universities Can Evaluate the Research Impact of Open Humanities for Societal Benefit

Authors : Paul Longley Arthur, Lydia Hearn

During the twenty-first century, for the first time, the volume of digital data has surpassed the amount of analog data. As academic practices increasingly become digital, opportunities arise to reshape the future of scholarly communication through more accessible, interactive, open, and transparent methods that engage a far broader and more diverse public.

Yet despite these advances, the research performance of universities and public research institutes remains largely evaluated through publication and citation analysis rather than by public engagement and societal impact.

This article reviews how changes to bibliometric evaluations toward greater use of altmetrics, including social media mentions, could enhance uptake of open scholarship in the humanities.

In addition, the article highlights current challenges faced by the open scholarship movement, given the complexity of the humanities in terms of its sources and outputs that include monographs, book chapters, and journals in languages other than English; the use of popular media not considered as scholarly papers; the lack of time and energy to develop digital skills among research staff; problems of authority and trust regarding the scholarly or non-academic nature of social media platforms; the prestige of large academic publishing houses; and limited awareness of and familiarity with advanced digital applications.

While peer review will continue to be a primary method for evaluating research in the humanities, a combination of altmetrics and other assessment of research impact through different data sources may provide a way forward to ensure the increased use, sustainability, and effectiveness of open scholarship in the humanities.

DOI : https://doi.org/10.3998/jep.788

Catégories
EN

Awareness Mentality and Strategic Behavior in Science

Author : Rafael Ball

Acknowledgement of scientific achievements was and is essentially achieved through the citation of a publication. Increasingly, however, it is no longer just the publication itself that plays an important role, but also the degree of attention that a scientist achieves with this very publication.

Thus, the importance of strategic behavior in science is progressing and an awareness mentality is spreading. In this paper, the causes and backgrounds of this development are discussed, identifying the use of reductionist, quantitative systems in science management and research funding, the loss of critical judgment and technocratic dominance, quantitative assessments used for decision making, altmetrics and the like as alternative views, the use of perception scores in reference databases and universities as well as ambitions of journals as main drivers.

Besides, different forms of strategic behavior in science and the resulting consequences and impacts are being highlighted.

URL : Awareness Mentality and Strategic Behavior in Science

DOI : https://doi.org/10.3389/frma.2021.703159

Catégories
EN

TeamTree analysis: A new approach to evaluate scientific production

Author : Frank W. Pfrieger

Advances in science and technology depend on the work of research teams and the publication of results through peer-reviewed articles representing a growing socio-economic resource. Current methods to mine the scientific literature regarding a field of interest focus on content, but the workforce credited by authorship remains largely unexplored.

Notably, appropriate measures of scientific production are debated. Here, a new bibliometric approach named TeamTree analysis is introduced that visualizes the development and composition of the workforce driving a field.

A new citation-independent measure that scales with the H index estimates impact based on publication record, genealogical ties and collaborative connections.

This author-centered approach complements existing tools to mine the scientific literature and to evaluate research across disciplines.

URL : TeamTree analysis: A new approach to evaluate scientific production

DOI : https://doi.org/10.1371/journal.pone.0253847

Catégories
EN

Which aspects of the Open Science agenda are most relevant to scientometric research and publishing? An opinion paper

Authors : Lutz Bornmann, Raf Guns, Michael Thelwall, Dietmar Wolfram

Open Science is an umbrella term that encompasses many recommendations for possible changes in research practices, management, and publishing with the objective to increase transparency and accessibility.

This has become an important science policy issue that all disciplines should consider. Many Open Science recommendations may be valuable for the further development of research and publishing but not all are relevant to all fields.

This opinion paper considers the aspects of Open Science that are most relevant for scientometricians, discussing how they can be usefully applied.

DOI : https://doi.org/10.1162/qss_e_00121

Catégories
EN

Conjoint analysis of researchers’ hidden preferences for bibliometrics, altmetrics, and usage metrics

Authors : Steffen Lemke, Athanasios Mazarakis, Isabella Peters

The amount of annually published scholarly articles is growing steadily, as is the number of indicators through which impact of publications is measured. Little is known about how the increasing variety of available metrics affects researchers’ processes of selecting literature to read.

We conducted ranking experiments embedded into an online survey with 247 participating researchers, most from social sciences. Participants completed series of tasks in which they were asked to rank fictitious publications regarding their expected relevance, based on their scores regarding six prototypical metrics.

Through applying logistic regression, cluster analysis, and manual coding of survey answers, we obtained detailed data on how prominent metrics for research impact influence our participants in decisions about which scientific articles to read.

Survey answers revealed a combination of qualitative and quantitative characteristics that researchers consult when selecting literature, while regression analysis showed that among quantitative metrics, citation counts tend to be of highest concern, followed by Journal Impact Factors.

Our results suggest a comparatively favorable view of many researchers on bibliometrics and widespread skepticism toward altmetrics.

The findings underline the importance of equipping researchers with solid knowledge about specific metrics’ limitations, as they seem to play significant roles in researchers’ everyday relevance assessments.

URL : Conjoint analysis of researchers’ hidden preferences for bibliometrics, altmetrics, and usage metrics

DOI : https://doi.org/10.1002/asi.24445

Catégories
FR

L’effet SIGAPS : La recherche médicale française sous l’emprise de l’évaluation comptable

Auteurs/Authors : Yves Gingras, Mahdi Khelfaoui

Cette recherche a pour but de mettre en évidence les effets pervers générés par l’introduction du système SIGAPS (Système d’interrogation, de gestion, et d’analyse des publications scientifiques) sur la production scientifique française en médecine et en sciences biomédicales.

Cet outil biblio-métrique de gestion et de financement de la recherche présente un exemple emblématique des dé-rives que peuvent générer les méthodes d’évaluation de la recherche reposant sur des critères pu-rement comptables.

Dans cette note, nous présentons d’abord le fonctionnement de SIGAPS, pour ensuite expliquer précisément en quoi les méthodes de calcul des « points SIGAPS », fondés sur les facteurs d’impact des revues et l’ordre des noms des co-auteurs, posent de nombreux problèmes.

Nous identifions notamment les effets du système SIGAPS sur les dynamiques de publications, les choix des lieux de publications, la langue de publication et les critères de recrutement et de promotion des chercheurs.

Finalement, nous montrons que l’utilisation du système SIGAPS ne répond pas bien à tous les critères de ce que l’on pourrait appeler une « éthique de l’évaluation » qui devrait respecter certaines règles, comme la transparence, l’équité et la validité des indicateurs.

URL : https://cirst2.openum.ca/files/sites/179/2020/10/Note_2020-05vf.pdf

Catégories
Non classé

A validation of coauthorship credit models with empirical data from the contributions of PhD candidates

Author : Paul Donner

A perennial problem in bibliometrics is the appropriate distribution of authorship credit for coauthored publications. Several credit allocation methods and formulas have been introduced, but there has been little empirical validation as to which method best reflects the typical contributions of coauthors.

This paper presents a validation of credit allocation methods using a new data set of author-provided percentage contribution figures obtained from the coauthored publications in cumulative PhD theses by authors from three countries that contain contribution statements.

The comparison of allocation schemes shows that harmonic counting performs best and arithmetic and geometric counting also perform well, while fractional counting and first author counting perform relatively poorly.

URL : https://www.mitpressjournals.org/doi/abs/10.1162/qss_a_00048?af=R