Archives des mots-clés : Serge P. J. M. Horbach

Open Science at the Generative AI Turn: An Exploratory Analysis of Challenges and Opportunities

Posté le par

Authors : Mohammad Hosseini, Serge P.J.M. Horbach, Kristi L. Holmes, Tony Ross-Hellauer

Technology influences Open Science (OS) practices, because conducting science in transparent, accessible, and participatory ways requires tools/platforms for collaborative research and sharing results. Due to this direct relationship, characteristics of employed technologies directly impact OS objectives. Generative Artificial Intelligence (GenAI) models are increasingly used by researchers for tasks such as text refining, code generation/editing, reviewing literature, data curation/analysis.

GenAI promises substantial efficiency gains but is currently fraught with limitations that could negatively impact core OS values such as fairness, transparency and integrity, and harm various social actors. In this paper, we explore possible positive and negative impacts of GenAI on OS.

We use the taxonomy within the UNESCO Recommendation on Open Science to systematically explore the intersection of GenAI and OS. We conclude that using GenAI could advance key OS objectives by further broadening meaningful access to knowledge, enabling efficient use of infrastructure, improving engagement of societal actors, and enhancing dialogue among knowledge systems.

However, due to GenAI limitations, it could also compromise the integrity, equity, reproducibility, and reliability of research, while also having potential implications for the political economy of research and its infrastructure. Hence, sufficient checks, validation and critical assessments are essential when incorporating GenAI into research workflows.

URL : Open Science at the Generative AI Turn: An Exploratory Analysis of Challenges and Opportunities

DOI : https://doi.org/10.31235/osf.io/zns7g

Additional experiments required: A scoping review of recent evidence on key aspects of Open Peer Review

Posté le par

Authors : Tony Ross-Hellauer, Serge P.J.M. Horbach

Diverse efforts are underway to reform the journal peer review system. Combined with growing interest in Open Science practices, Open Peer Review (OPR) has become of central concern to the scholarly community. However, what OPR is understood to encompass and how effective some of its elements are in meeting the expectations of diverse communities, are uncertain.

This scoping review updates previous efforts to summarize research on OPR to May 2022. Following the PRISMA methodological framework, it addresses the question: “What evidence has been reported in the scientific literature from 2017 to May 2022 regarding uptake, attitudes, and efficacy of two key aspects of OPR (Open Identities and Open Reports)?”

The review identifies, analyses and synthesizes 52 studies matching inclusion criteria, finding that OPR is growing, but still far from common practice. Our findings indicate positive attitudes towards Open Reports and more sceptical approaches to Open Identities.

Changes in reviewer behaviour seem limited and no evidence for lower acceptance rates of review invitations or slower turnaround times is reported in those studies examining those issues. Concerns about power dynamics and potential backfiring on critical reviews are in need of further experimentation.

We conclude with an overview of evidence gaps and suggestions for future research. Also, we discuss implications for policy and practice, both in the scholarly communications community and the research evaluation community more broadly.

URL : Additional experiments required: A scoping review of recent evidence on key aspects of Open Peer Review

DOI : https://doi.org/10.1093/reseval/rvae004

Pandemic Publishing: Medical journals drastically speed up their publication process for Covid-19

Posté le par

Author : Serge P.J.M. Horbach

In times of public crises, including the current Covid-19 pandemic, rapid dissemination of relevant scientific knowledge is of paramount importance. The duration of scholarly journals’ publication process is one of the main factors hindering quick delivery of new information.

While proper editorial assessment and peer review obviously require some time, turnaround times for medical journals can be up to several months, which is undesirable in the era of a crisis. Following initiatives of medical journals and scholarly publishers to accelerate their publication process, this study assesses whether medical journals have indeed managed to speed up their publication process for Covid-19 related articles.

It studies the duration of 14 medical journals’ publication process both during and prior to the current pandemic. Assessing a total of 669 articles, the study concludes that medical journals have indeed drastically accelerated the publication process for Covid-19 related articles since the outbreak of the pandemic. Compared to articles published in the same journals before the pandemic, turnaround times have decreased on average by 49%.

The largest decrease in number of days between submission and publication of articles was due to a decrease in the number of days required for peer review. For articles not related to Covid-19, no acceleration of the publication process is found.

While the acceleration of journals’ publication process is laudable from the perspective of quick information dissemination, it also raises concerns relating to the quality of the peer review process and the quality of the resulting publications.

URL : https://doi.org/10.1101/2020.04.18.045963

On the Willingness to Report and the Consequences of Reporting Research Misconduct: The Role of Power Relations

Posté le par

Authors : Serge P. J. M. Horbach, Eric Breit, Willem Halffman, Svenn-Erik Mamelund

While attention to research integrity has been growing over the past decades, the processes of signalling and denouncing cases of research misconduct remain largely unstudied.

In this article, we develop a theoretically and empirically informed understanding of the causes and consequences of reporting research misconduct in terms of power relations.

We study the reporting process based on a multinational survey at eight European universities (N = 1126). Using qualitative data that witnesses of research misconduct or of questionable research practices provided, we aim to examine actors’ rationales for reporting and not reporting misconduct, how they report it and the perceived consequences of reporting.

In particular we study how research seniority, the temporality of work appointments, and gender could impact the likelihood of cases being reported and of reporting leading to constructive organisational changes.

Our findings suggest that these aspects of power relations play a role in the reporting of research misconduct. Our analysis contributes to a better understanding of research misconduct in an academic context.

Specifically, we elucidate the processes that affect researchers’ ability and willingness to report research misconduct, and the likelihood of universities taking action.

Based on our findings, we outline specific propositions that future research can test as well as provide recommendations for policy improvement.

URL : On the Willingness to Report and the Consequences of Reporting Research Misconduct: The Role of Power Relations

DOI : https://doi.org/10.1007/s11948-020-00202-8