How unpredictable is research impact? Evidence from the UK’s Research Excellence Framework

Authors : Ohid Yaqub, Dmitry Malkov, Josh Siepel

Although ex post evaluation of impact is increasingly common, the extent to which research impacts emerge largely as anticipated by researchers, or as the result of serendipitous and unpredictable processes, is not well understood.

In this article, we explore whether predictions of impact made at the funding stage align with realized impact, using data from the UK’s Research Excellence Framework (REF). We exploit REF impact cases traced back to research funding applications, as a dataset of 2,194 case–grant pairs, to compare impact topics with funder remits.

For 209 of those pairs, we directly compare their descriptions of ex ante and ex post impact. We find that impact claims in these case–grant pairs are often congruent with each other, with 76% showing alignment between anticipated impact at funding stage and the eventual claimed impact in the REF. Co-production of research, often perceived as a model for impactful research, was a feature of just over half of our cases.

Our results show that, contrary to other preliminary studies of the REF, impact appears to be broadly predictable, although unpredictability remains important. We suggest that co-production is a reasonably good mechanism for addressing the balance of predictable and unpredictable impact outcomes.

URL : How unpredictable is research impact? Evidence from the UK’s Research Excellence Framework

DOI : https://doi.org/10.1093/reseval/rvad019

Enquête quantitative sur les pratiques et les besoins des chercheurs sur la gestion des données de la recherche, algorithmes et codes sources dans les établissements du site toulousain

Authors : Danielle Brunet, Soraya Demay, Pierre Diaz, Borbala Goncz, Laure Leclerc, Flora Poupinot, Sibilla Michelle

Le Comité de réflexion pour le partage et la valorisation des données de la recherche et la coordination de la Science Ouverte (CéSO) de l’Université de Toulouse a réalisé une enquête quantitative sur la gestion des données de la recherche, algorithmes et codes sources.

Adressée à l’ensemble de la communauté scientifique du site toulousain, son objectif était de produire un état des lieux des pratiques, des connaissances et des besoins des chercheurs en matière de gestion des données de la recherche. Les résultats permettront de préciser l’offre de services proposée sur le site toulousain.

Cette enquête concerne les établissements membres de l’Université de Toulouse ainsi que les organismes de recherche partenaires : Université Toulouse Capitole, Université Toulouse – Jean Jaurès, Université Toulouse III – Paul Sabatier, Institut national polytechnique de Toulouse (Toulouse INP), Institut national des sciences appliquées de Toulouse (INSA Toulouse), Institut supérieur de l’aéronautique et de l’espace (ISAE-SUPAERO), Institut national universitaire Champollion (INU Champollion), École nationale de l’aviation civile (ENAC), École nationale d’ingénieurs de Tarbes (ENIT), École nationale supérieure d’architecture de Toulouse (ENSA Toulouse), École nationale vétérinaire de Toulouse (ENVT), École nationale supérieure de formation de l’enseignement agricole (ENSFEA), Institut catholique d’arts et métiers (ICAM), École nationale supérieure des mines d’Albi-Carmaux (IMT Mines d’Albi), Toulouse Business School (TBS), Centre national d’études spatiales (CNES), Centre national de la recherche scientifique (CNRS), Institut national de recherche pour l’agriculture, l’alimentation et l’environnement (INRAE), Institut national de l’a santé et de la recherche médicale (Inserm), Institut de recherche pour le développement (IRD) ; Office national d’études et de recherche aérospatiales (Onera), Météo-France.

URL : Enquête quantitative sur les pratiques et les besoins des chercheurs sur la gestion des données de la recherche, algorithmes et codes sources dans les établissements du site toulousain

Original location : https://ut3-toulouseinp.hal.science/hal-04262708v1/

The use of shadow libraries at Universitas Indonesia

Authors : Fauzan Eka Kusuma, Rahmi Rahmi

Shadow libraries (SLs), such as Sci-Hub, Z-Library, and Library Genesis (LibGen), are online databases that provide content that is otherwise difficult to access (due to paywalls or other copyright controls) using unofficial methods of questionable legality. Interest in the SL phenomenon has focused on copyright infringement that occurs when a database provides library materials, for which access rights need to be purchased, without the knowledge of a given copyright owner.

This study analyzes the use of SLs at the Universitas Indonesia (UI). The research uses a quantitative approach, with a survey distributed to 262 undergraduate students at UI. The frequency of SL use in academic activities of UI students is compared with the use of the UI Online Public Access Catalog (OPAC UI).

The results show that most UI students have not used SLs. However, those who have used SLs report more positive impressions and higher levels of satisfaction compared with OPAC UI.

URL : The use of shadow libraries at Universitas Indonesia

DOI : https://doi.org/10.5210/fm.v28i10.12947

Pour quelles revues et publishers les chercheurs d’une université pluridisciplinaire travaillent-ils ?

Auteur/Author :  Maxence Larrieu

Le document contextualise et présente une cartographie réalisée en 2022 à Université Paris Cité, dans le cadre de la plateforme éditoriale OPUS. Les activités éditoriales visées sont celles nécessaires au fonctionnement d’une revue scientifique, comme rédacteur en chef, relecteur ou plus largement membre des comités éditoriaux. La cartographie décrit les revues dans lesquelles les chercheurs de l’université exercent ces activités.

Elle se démarque par l’utilisation du récent outil OpenEditors, qui partage les données structurées des comités éditoriaux des principaux publishers. Cette méthode, enrichie manuellement, autorise une vue omnidisciplinaire appropriée pour l’université : environ 320 revues sont identifiées, 130 pour la santé, idem pour les SHS et moitié moins pour les sciences et techniques.

Les revues sont ensuite analysées manuellement selon différents critères : gouvernance, plateforme, publisher commercial et modèle économique. Les résultats, composés de 7 graphiques, sont précédés d’une explication de la méthode et du jeu de données réalisé. Enfin, des exemples d’utilisation de la cartographie sont proposés.

URL : https://hal.science/hal-04197253

Is scientific knowledge socially constructed? A Bayesian account of Laboratory Life

Author : Henry Small

In the book Laboratory Life Latour and Woolgar present an account of how scientific “facts” are formed through a process of microsocial interactions among individuals and “inscription devices” in the lab initially described as social construction.

The process moves through a series of steps during which the details and nature of the object become more and more certain until all qualifications are dropped, and the “fact” emerges as secure scientific knowledge. An alternative to this account is described based on a Bayesian probabilistic framework which arrives at the same end point.

The motive force for the constructivist approach appears to involve social processes of convincing colleagues while the Bayesian approach relies on the consistency of theory and evidence as judged by the participants.

The role of social processes is discussed in Bayesian terms, the acquisition and asymmetry of information, and its analogy to puzzle solving. Some parallels between the Bayesian and constructivist accounts are noted especially in relation to information theory.

URL : Is scientific knowledge socially constructed? A Bayesian account of Laboratory Life

DOI : https://doi.org/10.3389/frma.2023.1214512

Assessing the Value of Subscription Journal Packages and Open Access Journal Articles in a Community College Context

Authors : Tim Dolan, Duncan Claflin

Introduction

Academic libraries are facing two trends: the rising cost of journal subscriptions—the ongoing ‘serials crisis’—and the increasing availability of scholarly journal articles through free, open access channels.

Methods

This study analyzed the bibliographies from 116 research projects created by students across disciplines at a small community college, and classified citations to journal articles based on the simplest way students could have accessed them.

Results

more than 60% of the scholarly journal articles that students used are available to them for free, while an additional 18% are available through state-sponsored database subscriptions available to any Massachusetts resident. Only 17% of the journal articles used were available to students solely through the library’s database packages, and more than half of those articles were used by students in the ADN nursing program.

Discussion

These findings call into question the value of expensive subscription databases to community college libraries and suggest a pattern of diminishing return on investment and value. The trend towards cancellation of ‘big deal’ journal packages at large institutions addresses similar concerns, though the dynamics are somewhat different at small institutions.

Conclusion

The paper concludes by suggesting some steps that small academic libraries might take to adapt to this changing publishing environment, in the areas of collection development and information literacy instruction, in order to better serve students.

URL : Assessing the Value of Subscription Journal Packages and Open Access Journal Articles in a Community College Context

DOI : https://doi.org/10.31274/jlsc.15673

Monitoring Organisational Article Processing Charges (APCs) using Bibliographic Information Sources: Turku University Library Case

Authors : Anna-Kaarina Linna, Irene Ylönen, Anna Salmi

As open access publishing has become more widespread and required by research funders and the research community, the management and monitoring of article processing charges (APCs) have emerged as an important task in research organisations around the world. Within this tendency, a question of the comprehensiveness of organisational APC monitoring has become relevant.

This case study demonstrates how the comprehensiveness of in-house APC monitoring can be evaluated using international bibliographic information sources like Web of Science and Scopus, where it is possible to identify the corresponding author, as well as Unpaywall and DOAJ, which contain information about the open access statuses and APCs of articles.

Based on study results, it can be assumed that the organisation’s in-house bookkeeping has succeeded in registering 52 percent of APC invoices while 48 percent have not been identified.

The results show that the number of unreported publications that have been openly published and whose corresponding author is affiliated with the university is almost equal to those registered in the university’s institutional APC report.

The study describes the stages of data collection and processing in order of implementation, which allows a similar review to be feasible in another organisation.

At the end of the article, development proposals are presented for both the organisations’ in-house data collection and the content of publishers’ invoices.

URL : Monitoring Organisational Article Processing Charges (APCs) using Bibliographic Information Sources: Turku University Library Case

DOI : https://doi.org/10.53377/lq.13361