Enriching research quality: A proposition for stakeholder heterogeneity

Authors : Thomas Franssen

Dominant approaches to research quality rest on the assumption that academic peers are the only relevant stakeholders in its assessment. In contrast, impact assessment frameworks recognize a large and heterogeneous set of actors as stakeholders. In transdisciplinary research non-academic stakeholders are actively involved in all phases of the research process and actor-network theorists recognize a broad and heterogeneous set of actors as stakeholders in all types of research as they are assigned roles in the socio-material networks, also termed ‘problematizations’, that researchers reconfigure.

Actor-network theorists consider research as a performative act that changes the reality of the stakeholders it, knowingly or unknowingly, involves. Established approaches to, and notions of, research quality do not recognize the heterogeneity of relevant stakeholders nor allow for reflection on the performative effects of research.

To enrich the assessment of research quality this article explores the problematization as a potential new object of evaluation. Problematizations are proposals for how the future might look. Hence, their acceptance does not only concern fellow academics but also all other human and other-than-human actors that figure in them.

To enrich evaluative approaches, this article argues for the inclusion of stakeholder involvement and stakeholder representation as dimensions of research quality.

It considers a number of challenges to doing so including the identification of stakeholders, developing quality criteria for stakeholder involvement and stakeholder representation, and the possibility of participatory research evaluation. It can alternatively be summarized as raising the question: for whose benefit do we conduct evaluations of research quality?

URL : Enriching research quality: A proposition for stakeholder heterogeneity

DOI : https://doi.org/10.1093/reseval/rvac012

Science and its significant other: Representing the humanities in bibliometric scholarship

Authors : Thomas Franssen, Paul Wouters

The cognitive and social structures, and publication practices, of the humanities have been studied bibliometrically for the past 50 years. This article explores the conceptual frameworks, methods, and data sources used in bibliometrics to study the nature of the humanities, and its differences and similarities in comparison with other scientific domains.

We give a historical overview of bibliometric scholarship between 1965 and 2018 that studies the humanities empirically and distinguishes between two periods in which the configuration of the bibliometric system differs remarkably.

The first period, 1965 to the 1980s, is characterized by bibliometric methods embedded in a sociological theoretical framework, the development and use of the Price Index, and small samples of journal publications from which references are used as data sources.

The second period, the 1980s to the present day, is characterized by a new intellectual hinterland—that of science policy and research evaluation—in which bibliometric methods become embedded.

Here metadata of publications becomes the primary data source with which publication profiles of humanistic scholarly communities are analyzed. We unpack the differences between these two periods and critically discuss the analytical avenues that different approaches offer.

URL : Science and its significant other: Representing the humanities in bibliometric scholarship

DOI : https://doi.org/10.1002/asi.24206

Scholars on Twitter: who and how many are they?

Authors :  Rodrigo Costas, Jeroen van Honk, Thomas Franssen

In this paper we present a novel methodology for identifying scholars with a Twitter account. By combining bibliometric data from Web of Science and Twitter users identified by Altmetric.com we have obtained the largest set of individual scholars matched with Twitter users made so far.

Our methodology consists of a combination of matching algorithms, considering different linguistic elements of both author names and Twitter names; followed by a rule-based scoring system that weights the common occurrence of several elements related with the names, individual elements and activities of both Twitter users and scholars matched.

Our results indicate that about 2% of the overall population of scholars in the Web of Science is active on Twitter. By domain we find a strong presence of researchers from the Social Sciences and the Humanities. Natural Sciences is the domain with the lowest level of scholars on Twitter.

Researchers on Twitter also tend to be younger than those that are not on Twitter. As this is a bibliometric-based approach, it is important to highlight the reliance of the method on the number of publications produced and tweeted by the scholars, thus the share of scholars on Twitter ranges between 1% and 5% depending on their level of productivity. Further research is suggested in order to improve and expand the methodology.

URL : https://arxiv.org/abs/1712.05667