Archives des mots-clés : Tony Ross-Hellauer

The societal impact of Open Science: a scoping review

Posté le par

Authors : Nicki Lisa Cole, Eva Kormann, Thomas Klebel, Simon Apartis, Tony Ross-Hellauer

Open Science (OS) aims, in part, to drive greater societal impact of academic research. Government, funder and institutional policies state that it should further democratize research and increase learning and awareness, evidence-based policy-making, the relevance of research to society’s problems, and public trust in research. Yet, measuring the societal impact of OS has proven challenging and synthesized evidence of it is lacking.

This study fills this gap by systematically scoping the existing evidence of societal impact driven by OS and its various aspects, including Citizen Science (CS), Open Access (OA), Open/FAIR Data (OFD), Open Code/Software and others. Using the PRISMA Extension for Scoping Reviews and searches conducted in Web of Science, Scopus and relevant grey literature, we identified 196 studies that contain evidence of societal impact. The majority concern CS, with some focused on OA, and only a few addressing other aspects.

Key areas of impact found are education and awareness, climate and environment, and social engagement. We found no literature documenting evidence of the societal impact of OFD and limited evidence of societal impact in terms of policy, health, and trust in academic research. Our findings demonstrate a critical need for additional evidence and suggest practical and policy implications.

URL : https://doi.org/10.1098/rsos.240286

Open Science at the Generative AI Turn: An Exploratory Analysis of Challenges and Opportunities

Posté le par

Authors : Mohammad Hosseini, Serge P.J.M. Horbach, Kristi L. Holmes, Tony Ross-Hellauer

Technology influences Open Science (OS) practices, because conducting science in transparent, accessible, and participatory ways requires tools/platforms for collaborative research and sharing results. Due to this direct relationship, characteristics of employed technologies directly impact OS objectives. Generative Artificial Intelligence (GenAI) models are increasingly used by researchers for tasks such as text refining, code generation/editing, reviewing literature, data curation/analysis.

GenAI promises substantial efficiency gains but is currently fraught with limitations that could negatively impact core OS values such as fairness, transparency and integrity, and harm various social actors. In this paper, we explore possible positive and negative impacts of GenAI on OS.

We use the taxonomy within the UNESCO Recommendation on Open Science to systematically explore the intersection of GenAI and OS. We conclude that using GenAI could advance key OS objectives by further broadening meaningful access to knowledge, enabling efficient use of infrastructure, improving engagement of societal actors, and enhancing dialogue among knowledge systems.

However, due to GenAI limitations, it could also compromise the integrity, equity, reproducibility, and reliability of research, while also having potential implications for the political economy of research and its infrastructure. Hence, sufficient checks, validation and critical assessments are essential when incorporating GenAI into research workflows.

URL : Open Science at the Generative AI Turn: An Exploratory Analysis of Challenges and Opportunities

DOI : https://doi.org/10.31235/osf.io/zns7g

Additional experiments required: A scoping review of recent evidence on key aspects of Open Peer Review

Posté le par

Authors : Tony Ross-Hellauer, Serge P.J.M. Horbach

Diverse efforts are underway to reform the journal peer review system. Combined with growing interest in Open Science practices, Open Peer Review (OPR) has become of central concern to the scholarly community. However, what OPR is understood to encompass and how effective some of its elements are in meeting the expectations of diverse communities, are uncertain.

This scoping review updates previous efforts to summarize research on OPR to May 2022. Following the PRISMA methodological framework, it addresses the question: “What evidence has been reported in the scientific literature from 2017 to May 2022 regarding uptake, attitudes, and efficacy of two key aspects of OPR (Open Identities and Open Reports)?”

The review identifies, analyses and synthesizes 52 studies matching inclusion criteria, finding that OPR is growing, but still far from common practice. Our findings indicate positive attitudes towards Open Reports and more sceptical approaches to Open Identities.

Changes in reviewer behaviour seem limited and no evidence for lower acceptance rates of review invitations or slower turnaround times is reported in those studies examining those issues. Concerns about power dynamics and potential backfiring on critical reviews are in need of further experimentation.

We conclude with an overview of evidence gaps and suggestions for future research. Also, we discuss implications for policy and practice, both in the scholarly communications community and the research evaluation community more broadly.

URL : Additional experiments required: A scoping review of recent evidence on key aspects of Open Peer Review

DOI : https://doi.org/10.1093/reseval/rvae004

The APC-Barrier and its effect on stratification in open access publishing

Posté le par

Authors : Thomas Klebel, Tony Ross-Hellauer

Current implementations of Open Access (OA) publishing frequently involve Article Publishing Charges (APCs). Increasing evidence emerges that APCs impede researchers with fewer resources in publishing their research OA.

We analysed 1.5 million scientific articles from journals listed in the Directory of Open Access Journals to assess average APCs and their determinants for a comprehensive set of journal publications, across scientific disciplines, world regions and through time.

Levels of APCs were strongly stratified by scientific fields and the institutions’ countries, corroborating previous findings on publishing cultures and the impact of mandates of research funders.

After controlling for country and scientific field with a multilevel mixture model, however, we found small to moderate effects of levels of institutional resourcing on the level of APCs.

Effects were largest in countries with low GDP, suggesting decreasing marginal effects of institutional resources when general levels of funding are high. Our findings provide further evidence on how APCs stratify OA publishing and highlight the need for alternative publishing models.

URL : The APC-Barrier and its effect on stratification in open access publishing

DOI : https://doi.org/10.1162/qss_a_00245

Indicators of research quality, quantity, openness and responsibility in institutional review, promotion and tenure policies across seven countries

Posté le par

Authors : Nancy Pontika, Thomas Klebel, Antonia Correia, Hannah Metzler, Petr Knoth, Tony Ross-Hellauer

The need to reform research assessment processes related to career advancement at research institutions has become increasingly recognised in recent years, especially to better foster open and responsible research practices. Current assessment criteria are believed to focus too heavily on inappropriate criteria related to productivity and quantity as opposed to quality, collaborative open research practices, and the socio-economic impact of research.

Evidence of the extent of these issues is urgently needed to inform actions for reform, however. We analyse current practices as revealed by documentation on institutional review, promotion and tenure processes in seven countries (Austria, Brazil, Germany, India, Portugal, United Kingdom and United States of America).

Through systematic coding and analysis of 143 RPT policy documents from 107 institutions for the prevalence of 17 criteria (including those related to qualitative or quantitative assessment of research, service to the institution or profession, and open and responsible research practices), we compare assessment practices across a range of international institutions to significantly broaden this evidence-base.

Although prevalence of indicators varies considerably between countries, overall we find that currently open and responsible research practices are minimally rewarded and problematic practices of quantification continue to dominate.

URL : Indicators of research quality, quantity, openness and responsibility in institutional review, promotion and tenure policies across seven countries

DOI : https://doi.org/10.1162/qss_a_00224

The APC-Effect: Stratification in Open Access Publishing

Posté le par

Authors : Thomas Klebel,Tony Ross-Hellauer

Current implementations of Open Access (OA) publishing frequently involve Article Publishing Charges (APCs). Increasing evidence emerges that APCs impede researchers with fewer resources in publishing their research OA.

We analysed 1.5 million scientific articles from journals listed in the Directory of Open Access Journals to assess average APCs and their determinants for a comprehensive set of journal publications, across scientific disciplines, world regions and through time. Levels of APCs were strongly stratified by scientific fields and the institutions’ countries, corroborating previous findings on publishing cultures and the impact of mandates of research funders.

After controlling for country and scientific field with a multilevel mixture model, however, we found small to moderate effects of levels of institutional resourcing on the level of APCs. This is what we call the APC-Effect.

Effects were largest in countries with low GDP, suggesting decreasing marginal effects of institutional resources when general levels of funding are high. Our findings provide further evidence on how APCs stratify OA publishing and highlight the need for alternative publishing models.

DOI : https://doi.org/10.31222/osf.io/w5szk

Between administration and research: Understanding data management practices in an institutional context

Posté le par

Authors : Stefan Reichmann, Thomas Klebel, Ilire Hasani-Mavriqi, Tony Ross-Hellauer

Research Data Management (RDM) promises to make research outputs more transparent, findable, and reproducible. Strategies to streamline data management across disciplines are of key importance.

This paper presents results of an institutional survey (N = 258) at a medium-sized Austrian university with a STEM focus, supplemented with interviews (N = 18), to give an overview of the state-of-play of RDM practices across faculties and disciplinary contexts.

RDM services are on the rise but remain somewhat behind leading countries like the Netherlands and UK, showing only the beginnings of a culture attuned to RDM. There is considerable variation between faculties and institutes with respect to data amounts, complexity of data sets, data collection and analysis, and data archiving.

Data sharing practices within fields tend to be inconsistent. RDM is predominantly regarded as an administrative task, to the detriment of considerations of good research practice. Problems with RDM fall in two categories: Generic problems transcend specific research interests, infrastructures, and departments while discipline-specific problems need a more targeted approach.

The paper extends the state-of-the-art on RDM practices by combining in-depth qualitative material with quantified, detailed data about RDM practices and needs. The findings should be of interest to any comparable research institution with a similar agenda.

URL : Between administration and research: Understanding data management practices in an institutional context

DOI : https://doi.org/10.1002/asi.24492