Do we achieve anything by teaching research integrity to starting PhD students?

Posté le par

Authors : Shila Abdi, Steffen Fieuws, Benoit Nemer, Kris Dierickx

Education of young researchers has been proposed as a way to promote research integrity. However, the effectiveness of research integrity education on PhD students is unknown. In a longitudinal design, we surveyed over 1000 starting PhD students from various disciplines regarding knowledge, attitude and behaviour before, immediately after and 3 months after a compulsory 3-h course given by a panel of experts.

Compared with a control group who did not follow the course, the course recipients showed significant (multivariate analysis) but modest improvements in knowledge and attitude scores immediately after the course, but not after 3 months; a prolonged impact was apparent regarding behaviour.

Moreover, the course spurred 93% of PhD students to have conversations about research integrity and 79% declared applying the content of the course. Among other interventions, formal education in research integrity may contribute to foster a climate of research integrity in academia.

URL : Do we achieve anything by teaching research integrity to starting PhD students?

DOI : https://doi.org/10.1057/s41599-021-00908-5

Does double-blind peer review reduce bias? Evidence from a top computer science conference

Posté le par

Authors : Mengyi Sun, Jainabou Barry Danfa, Misha Teplitskiy

Peer review is essential for advancing scientific research, but there are long-standing concerns that authors’ prestige or other characteristics can bias reviewers. Double-blind peer review has been proposed as a way to reduce reviewer bias, but the evidence for its effectiveness is limited and mixed.

Here, we examine the effects of double-blind peer review by analyzing the review files of 5,027 papers submitted to a top computer science conference that changed its reviewing format from single- to double-blind in 2018.

First, we find that the scores given to the most prestigious authors significantly decreased after switching to double-blind review. However, because many of these papers were above the threshold for acceptance, the change did not affect paper acceptance significantly.

Second, the inter-reviewer disagreement increased significantly in the double-blind format.

Third, papers rejected in the single-blind format are cited more than those rejected under double-blind, suggesting that double-blind review better excludes poorer quality papers.

Lastly, an apparently unrelated change in the rating scale from 10 to 4 points likely reduced prestige bias significantly such that papers’ acceptance was affected.

These results support the effectiveness of double-blind review in reducing biases, while opening new research directions on the impact of peer-review formats.

URL : Does double-blind peer review reduce bias? Evidence from a top computer science conference

DOI : https://doi.org/10.1002/asi.24582

Identifying and characterizing social media communities: a socio-semantic network approach to altmetrics

Posté le par

Authors : Wenceslao Arroyo-Machado, Daniel Torres-Salinas, Nicolas Robinson-Garcia

Altmetric indicators allow exploring and profiling individuals who discuss and share scientific literature in social media. But it is still a challenge to identify and characterize communities based on the research topics in which they are interested as social and geographic proximity also influence interactions.

This paper proposes a new method which profiles social media users based on their interest on research topics using altmetric data. Social media users are clustered based on the topics related to the research publications they share in social media.

This allows removing linkages which respond to social or personal proximity and identifying disconnected users who may have similar research interests. We test this method for users tweeting publications from the fields of Information Science & Library Science, and Microbiology.

We conclude by discussing the potential application of this method and how it can assist information professionals, policy managers and academics to understand and identify the main actors discussing research literature in social media.

URL : Identifying and characterizing social media communities: a socio-semantic network approach to altmetrics

DOI : https://doi.org/10.1007/s11192-021-04167-8

L’usage de la plateforme HAL par les unités de recherche de l’Université de Lille. La situation en 2021

Posté le par

Auteurs/Authors : Eric Kergosien, Joachim Schöpfel

Cette note présente les résultats d’une étude de suivi sur l’usage de la plateforme HAL par les laboratoires de recherche de l’Université de Lille, réalisée en avril 2021. L’analyse a porté sur les dépôts dans HAL, sur la création d’une collection et sur la part des documents en libre accès.

L’étude propose une photographie de la situation en 2021, par rapport aux résultats des analyses de 2019 et 2020, en montrant l’évolution de l’usage de HAL par les unités de recherche et l’impact de la mise en place d’une archive institutionnelle locale nommée LillOA.

URL : https://hal.archives-ouvertes.fr/hal-03372596

Dissimuler ou disséminer ? Une étude sur le sort réservé aux résultats négatifs

Posté le par

Auteures/Authors : Marie-Emilia Herbet, Jérémie Leonard, Maria Santangelo, Lucie Albaret

Une enquête composée de 34 questions a été adressée à des chercheurs en chimie, physique, sciences de l’ingénieur et de l’environnement, en vue d’identifier leur rapport aux résultats de recherche infructueux ainsi que les freins et les leviers de leur diffusion.

L’étude se fonde sur 310 réponses complètes émanant de participants affiliés à des établissements de recherche et d’enseignement français. Menée dans le cadre du projet Datacc, porté par les bibliothèques universitaires de Lyon et Grenoble Alpes, engagées dans l’accompagnement des chercheurs à l’ouverture des données de recherche, notre étude permet de combler le déficit de données sur le sujet au regard des disciplines concernées.

Elle relève que 81% des chercheurs interrogés ont déjà produit des résultats négatifs pertinents et 75% se disent prêts à publier ce type de données. Pourtant, seuls 12,5% des répondants ont déjà eu l’occasion de le faire dans une revue scientifique. Ce contraste béant entre l’intention et la pratique soulève des interrogations sur les obstacles en présence et les solutions potentielles à apporter.

URL : Dissimuler ou disséminer ? Une étude sur le sort réservé aux résultats négatifs

Original location : https://hal.archives-ouvertes.fr/hal-03371040

Science rules! A qualitative study of scientists’ approaches to grant lottery

Posté le par

Author : Axel Philipps

Using peer review to assess the validity of research proposals has always had its fair share of critics, including a more-than-fair-share of scholars. The debate about this method of assessing these proposals now seems trivial when compared with assessing the validity for granting funding by lottery.

Some of the same scholars have suggested that the way grant lottery was being assessed has made random allocation seem even-handed, less biased and more supportive of innovative research.

But we know little of what researchers actually think about grant lottery and even less about the thoughts of those scientists who rely on funding. This paper examines scientists’ perspectives on selecting grants by ‘lots’ and how they justify their support or opposition.

How do they approach something scientifically that is, in itself, not scientific? These approaches were investigated with problem-centered interviews conducted with natural scientists in Germany.

The qualitative interviews for this paper reveal that scientists in dominated and dominating field positions are, more or less, open to the idea of giving a selection process by lots a try. Nonetheless, they are against pure randomization because from their point of view it is incompatible with scientific principles.

They rather favor a combination of grant lottery and peer review processes, assuming that only under these conditions could randomly allocated funding be an integral and legitimate part of science.

URL : Science rules! A qualitative study of scientists’ approaches to grant lottery

DOI : https://doi.org/10.1093/reseval/rvaa027

The role of metrics in peer assessments

Posté le par

Authors :  Liv Langfeldt, Ingvild Reymert, Dag W Aksnes

Metrics on scientific publications and their citations are easily accessible and are often referred to in assessments of research and researchers. This paper addresses whether metrics are considered a legitimate and integral part of such assessments. Based on an extensive questionnaire survey in three countries, the opinions of researchers are analysed.

We provide comparisons across academic fields (cardiology, economics, and physics) and contexts for assessing research (identifying the best research in their field, assessing grant proposals and assessing candidates for positions).

A minority of the researchers responding to the survey reported that metrics were reasons for considering something to be the best research. Still, a large majority in all the studied fields indicated that metrics were important or partly important in their review of grant proposals and assessments of candidates for academic positions.

In these contexts, the citation impact of the publications and, particularly, the number of publications were emphasized. These findings hold across all fields analysed, still the economists relied more on productivity measures than the cardiologists and the physicists. Moreover, reviewers with high scores on bibliometric indicators seemed more frequently (than other reviewers) to adhere to metrics in their assessments.

Hence, when planning and using peer review, one should be aware that reviewers—in particular reviewers who score high on metrics—find metrics to be a good proxy for the future success of projects and candidates, and rely on metrics in their evaluation procedures despite the concerns in scientific communities on the use and misuse of publication metrics.

URL : The role of metrics in peer assessments

DOI : https://doi.org/10.1093/reseval/rvaa032