EN

Responsible research evaluation: integrating quality, leadership, and integrity in national systems. The case of Peru

Posted on by Hans Dillaerts

Authors : Pablo Alejandro Millones-Gomez, Jean Paul Simon Castillo-Nunez, Kenyie Jossuet Paucca-Calla, Carlos Alberto Minchon-Medina, Vanessa Lizet Castro-Delgado, Samantha Sotelo-Llancari, Cecilia Ignacio-Punin, David Yeret Rodríguez-Salazar, Víctor Hugo Urrutia-Baca

Introduction:

National research evaluation systems often rely on publication-based metrics that equate productivity with performance while overlooking scientific leadership and research integrity. This study examines the Peruvian National Registry of Science, Technology, and Innovation (RENACYT) to inform a more multidimensional framework for research evaluation.

Materials and methods:

An observational, non-experimental, and analytical study was conducted using data from RENACYT, Scopus, and SciVal for 9,651 researchers during 2019–2024. Four dimensions were assessed across hierarchical levels: scientific production, journal-based impact (Q1–Q4), corresponding authorship as a proxy of leadership, and retractions as indicators of research integrity. Descriptive statistics, ANOVA, repeated-measures tests, and count regression models (Poisson, negative binomial, and zero-inflated specifications) were applied.

Results:

A total of 92,284 publications were identified. Productivity increased across RENACYT levels (3.9 publications in Level VII vs. 62.5 in Distinguished; F = 1,162.572, p < 0.001), although with substantial within-level dispersion and differentiated temporal trajectories (Time × Level: F = 44.662, p < 0.001). Higher levels concentrated Q1 output (30.8 vs. 0.7 articles per author; F = 1,090.183, p < 0.001), while differences became less pronounced in Q3–Q4 journals. Corresponding authorship increased with level (β = 1.624 for Level I, p < 0.001) but remained heterogeneous even among top categories. Retractions were positively associated with productivity (coef. = 0.013, p < 0.001) reflecting differential exposure to integrity-related risks rather than uniform patterns across levels.

Conclusion:

RENACYT captures gradients in productivity and quality but insufficiently differentiates leadership and integrity. These findings support the proposal of a hybrid evaluation framework integrating productivity with explicit recognition of intellectual leadership and research integrity.

URL : Responsible research evaluation: integrating quality, leadership, and integrity in national systems. The case of Peru

Original location : https://www.frontiersin.org/journals/research-metrics-and-analytics/articles/10.3389/frma.2026.1842222/full

EN

Faculty Perspectives on Open Access: A Multidimensional Analysis of Open-Access Usage and Library Relationships at an R2 Institution

Posted on by Hans Dillaerts

Authors : James E. Van Loon, Julia E. Rodriguez

Introduction: The focus of this study is the broad impact open-access (OA) publishing has on all aspects of faculty members’ work (including publishing habits, research activities, and teaching) and on faculty relationships with the library.

Methods: An online survey addressing the impact of OA publishing on faculty and on their relationship with the library was administered to full-time tenure-track faculty at an R2 institution. Responses were analyzed by disciplinary unit, and themes were developed from faculty qualitative responses.

Results: A majority of faculty respondents had experience with OA publishing, with payment of article processing charges (APCs) to OA publishers, and with the use of OA materials in research and teaching. Many faculty respondents reported a substantial impact of OA on their research, teaching, and authorship, but expressed concern with the funding of OA publishing fees, with the quality and reputation of OA journals, and with the ethics of the OA business model.

Discussion: Faculty respondents demonstrate differing levels of familiarity with OA materials in all three roles. Overall, they are more receptive to OA materials as readers and researchers and more demanding about the quality, impact, and cost of OA publishing as authors.

Conclusion: Despite recognition of the benefits of OA materials, faculty respondents express concern over the cost, quality, and ethics of OA publishing. Faculty respondents also indicate a desire for the institution to provide funding for OA publications and rely on the library for expertise in locating, evaluating, and utilizing OA publications.

URL : Faculty Perspectives on Open Access: A Multidimensional Analysis of Open-Access Usage and Library Relationships at an R2 Institution

DOI : https://doi.org/10.31274/jlsc.20328

EN

Shared responsibility to address questionable research practices? – A study of perceived efficacy of organizational research integrity policies

Posted on by Hans Dillaerts

Authors : Simon Fuglsang, Serge P.J.M. Horbach, Mads P. Sørensen, Niels Mejlgaard

Background

In response to widespread concerns about research integrity, recent years have seen numerous efforts to safeguard against research misconduct and questionable research practices. Research-performing organizations are among the key actors involved in implementing such efforts. However, little is known about the effectiveness of organizational policy initiatives.

Methods and materials

In this study, we investigate the ability of organizations to change researchers’ behavior through the perspective of the researcher. We analyze data from the International Research Integrity Survey (IRIS), a survey of researchers in Europe, Canada, Australia, and the USA. We specifically investigate whether researchers’ perceptions of their organizations’ research integrity policies relate to the degree to which they engage in questionable research practices.

Results

We find that awareness of policies, evaluations of the effectiveness of policies, as well as confidence in their organizations’ research integrity policies all relate to lower levels of engagement in questionable research practices. However, we also find that this relationship is highly sensitive to both researchers’ research integrity self-confidence and their general attitudes toward research integrity.

Conclusions

As such, while findings indicate that organizations’ policy efforts can influence researcher behavior, this influence is contingent on researcher acceptance and empowerment.

URL : Shared responsibility to address questionable research practices? – A study of perceived efficacy of organizational research integrity policies

DOI : https://doi.org/10.1080/08989621.2026.2679561

EN

Effectiveness of the researcher-led “Peerspectives” peer review training course on review quality, knowledge, and skills among doctoral students in the biomedical sciences: a pre-post study

Posted on by Hans Dillaerts

Authors : Jessica L. Rohmann, Nadja Wülk, Kerstin Rubarth, Hannah Grillmaier, Iman Abdikarim, Mariana Lopes Simões, Sara Schroter, Marco Piccininni, Tobias Kurth, Toivo Glatz

Background

Peer review remains a cornerstone of scientific knowledge dissemination, yet comprehensive, practically relevant training is limited. This inspired us to develop Peerspectives, a peer review training course for doctoral students in the biomedical sciences in Berlin, Germany. We aimed to assess the effectiveness of the Peerspectives course on editor-judged quality of peer review reports.

Methods

Doctoral students in health research fields who enrolled in the Peerspectives course between October 2020 and August 2022 were invited to participate in the study, and 80 consented. The ~18 week-long course provided training on the structure, purpose, and conduct of peer review and editorial processes in biomedical journals. It included 12 h of lectures, homework assignments, and 12 h of hands-on, small-group workshops, during which students reviewed original research manuscripts currently under consideration at The BMJ under the guidance of experienced mentors.

The primary outcome was the overall quality of the peer review reports as judged by two independent BMJ editors using the global score of the Review Quality Instrument (RQI) pre- and post-intervention. Additionally, we compared participants’ post-course scores with those of actual BMJ reviewers. We also compared participants’ self-assessed knowledge and skills related to scholarly peer review (1–5 Likert scale) before and after the course.

Results

After course completion, the editor-assessed overall quality of the participants’ peer review reports was higher than before the course (median increase of 0.5 points, p < 0.001; mean increase of 0.36 points, p < 0.001). The RQI scores of participants’ post-course reports were not non-inferior to those of actual BMJ reviewers for the same manuscripts. Self-assessed peer review-related knowledge skills increased across all questionnaire items after course completion. Greatest improvements were seen in understanding reviewer expectations (increase in means from 2.9 to 4.5), confidence in reviewing (2.5 to 3.9), and knowing what to look for while reviewing (2.8 to 4.2).

Conclusions

Providing doctoral students with comprehensive training resulted in an editorially significant increase in review report quality and improved understanding of the role and expectations of peer reviewers in the scholarly publishing processes and confidence in giving constructive feedback.

URL : Effectiveness of the researcher-led “Peerspectives” peer review training course on review quality, knowledge, and skills among doctoral students in the biomedical sciences: a pre-post study

DOI : https://doi.org/10.1186/s41073-026-00220-3

EN

Towards a more informed and balanced use of scientific performance metrics

Posted on by Hans Dillaerts

Authors :  Jaap J A Denissen, Klaas Sijtsma, Wil M P van der Aalst

The goal of scientific assessment is to predict which individuals can make optimal use of limited resources within a specific context to make optimal allocation decisions. In academic contexts that pertain to individual-level allocations, this is most relevant for decisions on whom to hire for academic positions, nominate for awards, or whose research projects to fund.

The current perspective paper draws upon insights from decades of psychometric research and more recent research on scientific performance to derive a set of five psychometric criteria that should be met for optimal assessment procedures in academia. Although data-driven decision making has gained popularity in most domains, there is increasing resistance against using quantitative measurements in scientific assessment.

Recently, several stakeholders have proposed to jettison such measurements and focus instead on qualitative indicators or narratives. We argue that both quantitative and qualitative assessment do not always meet our five criteria, but solely relying on qualitative indicators appears to be a suboptimal strategy.

We argue instead that there are smarter ways to use quantitative indicators so that they become more reliable, predictive, and ultimately also more efficient and equitable. We conclude with a set of recommendations for scientific quality assessment that is based on the most recent psychometric and scientific insights. In an appendix, we apply these recommendations to a Dutch case study of how researcher information is considered in the application procedure for a prestigious individual grant.

URL : Towards a more informed and balanced use of scientific performance metrics

DOI : https://doi.org/10.1093/reseval/rvag023

EN

Developing digital stories in research for science communication: reflections from researchers

Posted on by Hans Dillaerts

Authors : Leanne Townsend, Mikelis Grivins, Christina Noble, Claire Hardy, Livia Ortolani, Gusztáv Nemes, Marta Baena-Sanz, Mags Currie, Mar Delgado

Audiovisual communication methods such as digital storytelling can reach wide audiences to realise greater societal research impact. Increasingly, researchers embrace (or are expected to embrace) these approaches but often lack relevant skills.

This paper draws on Horizon Europe-funded research where digital stories were developed in 20 European regions. Findings from a survey completed by the researchers highlight skills- and engagement-based challenges and explore how capacity to develop digital stories was built.

The paper focuses on the role of digital storytelling in science communication, and the challenges researchers face in developing these outputs, including in ensuring meaningful participant involvement and the authentic representation of participants’ voices within the final narratives.

We discuss how to better support researchers to embrace digital storytelling as a science communication method, with recommendations for effective research impact.

URL : Developing digital stories in research for science communication: reflections from researchers

DOI : https://doi.org/10.22323/148520260131095010

EN

Modeling the impact of research data unavailability on science

Posted on by Hans Dillaerts

Authors : Jorge Chamorro-Padial, Francisco-Javier Rodrigo-Ginés, Rosa Rodrí­guez-Sánchez, R.M. Gil, Roberto García

Scientific progress depends on the accessibility and reproducibility of research outputs. Unfortunately, datasets and other referenced resources in academic publications frequently become unavailable over time, limiting reproducibility and reuse.

In this work, we quantitatively analyze the potential impact of research data unavailability by applying economic, probabilistic, and network based models to scientific citation networks. Rather than measuring knowledge directly, we use citation based network metrics as proxies for the dissemination and potential reuse of scientific results, and study how the absence of data-linked resources affects impact propagation and productivity-related indicators.

We further examine the resilience of citation networks under different modeling assumptions and analyze the role of highly influential nodes, or superpropagators, in amplifying the effects of dataset loss.

Our results reveal structural dependencies on vulnerable data sources and show that the magnitude of the impact depends strongly on network position and model assumptions.

These findings provide quantitative evidence of the systemic consequences of data unavailability and underline the importance of long-term data preservation and accessibility policies in scientific research.

URL : Modeling the impact of research data unavailability on science

DOI : https://doi.org/10.1016/j.joi.2026.101813