Archives des mots-clés : research integrity

RipetaScore: Measuring the Quality, Transparency, and Trustworthiness of a Scientific Work

Posté le par

Authors : Josh Q. Sumner, Cynthia Hudson Vitale, Leslie D. McIntosh

A wide array of existing metrics quantifies a scientific paper’s prominence or the author’s prestige. Many who use these metrics make assumptions that higher citation counts or more public attention must indicate more reliable, better quality science.

While current metrics offer valuable insight into scientific publications, they are an inadequate proxy for measuring the quality, transparency, and trustworthiness of published research.

Three essential elements to establishing trust in a work include: trust in the paper, trust in the author, and trust in the data. To address these elements in a systematic and automated way, we propose the ripetaScore as a direct measurement of a paper’s research practices, professionalism, and reproducibility.

Using a sample of our current corpus of academic papers, we demonstrate the ripetaScore’s efficacy in determining the quality, transparency, and trustworthiness of an academic work.

In this paper, we aim to provide a metric to evaluate scientific reporting quality in terms of transparency and trustworthiness of the research, professionalism, and reproducibility.

URL : RipetaScore: Measuring the Quality, Transparency, and Trustworthiness of a Scientific Work

DOI : https://doi.org/10.3389/frma.2021.751734

Do we achieve anything by teaching research integrity to starting PhD students?

Posté le par

Authors : Shila Abdi, Steffen Fieuws, Benoit Nemer, Kris Dierickx

Education of young researchers has been proposed as a way to promote research integrity. However, the effectiveness of research integrity education on PhD students is unknown. In a longitudinal design, we surveyed over 1000 starting PhD students from various disciplines regarding knowledge, attitude and behaviour before, immediately after and 3 months after a compulsory 3-h course given by a panel of experts.

Compared with a control group who did not follow the course, the course recipients showed significant (multivariate analysis) but modest improvements in knowledge and attitude scores immediately after the course, but not after 3 months; a prolonged impact was apparent regarding behaviour.

Moreover, the course spurred 93% of PhD students to have conversations about research integrity and 79% declared applying the content of the course. Among other interventions, formal education in research integrity may contribute to foster a climate of research integrity in academia.

URL : Do we achieve anything by teaching research integrity to starting PhD students?

DOI : https://doi.org/10.1057/s41599-021-00908-5

Good friend or good student? An interview study of perceived conflicts between personal and academic integrity among students in three European countries

Posté le par

Authors : Mads Paludan Goddiksen, Una Quinn, Nóra Kovács, Thomas Bøker Lund, Peter Sandøe, Orsolya Varga, Mikkel Willum Johansen

Students are often reluctant to report the academic dishonesty of their peers. Loyalty to friends and classmates has previously been identified as an important reason for this. This paper explores loyalty conflicts among students from upper secondary school, through bachelor’s, to Ph.D. level.

Drawing on semi-structured qualitative interviews (N = 72) conducted in Denmark, Ireland and Hungary, we show that loyalty considerations among students can be complex and draw on a range of norms including responsibility.

The study demonstrates how students are often willing to assume substantial personal responsibility for dealing with the academic dishonesty of a peer, often preferring this to reporting.

However, when deciding on the right course of action, they also perceive tensions between the norms of the good researcher and student and their own norms of being a good friend and person.

The loyalty considerations and tension were identified in all three countries and across the educational levels, which suggests that this is a cross-cultural challenge.

We argue that institutions should formally decide whether they want students to take some degree of responsibility themselves for addressing less serious cases of academic dishonesty and communicate their decision to their students.

URL : Good friend or good student? An interview study of perceived conflicts between personal and academic integrity among students in three European countries

DOI : https://doi.org/10.1080/08989621.2020.1826319

Honest signaling in academic publishing

Posté le par

Authors : Leonid Tiokhin, Karthik Panchanathan, Daniel Lakens, Simine Vazire, Thomas Morgan, Kevin Zollman

Academic journals provide a key quality-control mechanism in science. Yet, information asymmetries and conflicts of interests incentivize scientists to deceive journals about the quality of their research.

How can honesty be ensured, despite incentives for deception? Here, we address this question by applying the theory of honest signaling to the publication process. Our models demonstrate that several mechanisms can ensure honest journal submission, including differential benefits, differential costs, and costs to resubmitting rejected papers.

Without submission costs, scientists benefit from submitting all papers to high-ranking journals, unless papers can only be submitted a limited number of times. Counterintuitively, our analysis implies that inefficiencies in academic publishing (e.g., arbitrary formatting requirements, long review times) can serve a function by disincentivizing scientists from submitting low-quality work to high-ranking journals.

Our models provide simple, powerful tools for understanding how to promote honest paper submission in academic publishing.

URL : Honest signaling in academic publishing

DOI : https://doi.org/10.1371/journal.pone.0246675

Rethinking success, integrity, and culture in research (part 2) — a multi-actor qualitative study on problems of science

Posté le par

Authors : Noémie Aubert Bonn, Wim Pinxten

Background

Research misconduct and questionable research practices have been the subject of increasing attention in the past few years. But despite the rich body of research available, few empirical works also include the perspectives of non-researcher stakeholders.

Methods

We conducted semi-structured interviews and focus groups with policy makers, funders, institution leaders, editors or publishers, research integrity office members, research integrity community members, laboratory technicians, researchers, research students, and former-researchers who changed career to inquire on the topics of success, integrity, and responsibilities in science.

We used the Flemish biomedical landscape as a baseline to be able to grasp the views of interacting and complementary actors in a system setting.

Results

Given the breadth of our results, we divided our findings in a two-paper series with the current paper focusing on the problems that affect the integrity and research culture. We first found that different actors have different perspectives on the problems that affect the integrity and culture of research.

Problems were either linked to personalities and attitudes, or to the climates in which researchers operate. Elements that were described as essential for success (in the associate paper) were often thought to accentuate the problems of research climates by disrupting research culture and research integrity.

Even though all participants agreed that current research climates need to be addressed, participants generally did not feel responsible nor capable of initiating change. Instead, respondents revealed a circle of blame and mistrust between actor groups.

Conclusions

Our findings resonate with recent debates, and extrapolate a few action points which might help advance the discussion.

First, the research integrity debate must revisit and tackle the way in which researchers are assessed.

Second, approaches to promote better science need to address the impact that research climates have on research integrity and research culture rather than to capitalize on individual researchers’ compliance.

Finally, inter-actor dialogues and shared decision making must be given priority to ensure that the perspectives of the full research system are captured. Understanding the relations and interdependency between these perspectives is key to be able to address the problems of science.

URL : Rethinking success, integrity, and culture in research (part 2) — a multi-actor qualitative study on problems of science

DOI : https://doi.org/10.1186/s41073-020-00105-z

Rethinking success, integrity, and culture in research (part 1) — a multi-actor qualitative study on success in science

Posté le par

Authors : Noémie Aubert Bonn, Wim Pinxten

Background

Success shapes the lives and careers of scientists. But success in science is difficult to define, let alone to translate in indicators that can be used for assessment. In the past few years, several groups expressed their dissatisfaction with the indicators currently used for assessing researchers.

But given the lack of agreement on what should constitute success in science, most propositions remain unanswered. This paper aims to complement our understanding of success in science and to document areas of tension and conflict in research assessments.

Methods

We conducted semi-structured interviews and focus groups with policy makers, funders, institution leaders, editors or publishers, research integrity office members, research integrity community members, laboratory technicians, researchers, research students, and former-researchers who changed career to inquire on the topics of success, integrity, and responsibilities in science.

We used the Flemish biomedical landscape as a baseline to be able to grasp the views of interacting and complementary actors in a system setting.

Results

Given the breadth of our results, we divided our findings in a two-paper series, with the current paper focusing on what defines and determines success in science. Respondents depicted success as a multi-factorial, context-dependent, and mutable construct.

Success appeared to be an interaction between characteristics from the researcher (Who), research outputs (What), processes (How), and luck. Interviewees noted that current research assessments overvalued outputs but largely ignored the processes deemed essential for research quality and integrity.

Interviewees suggested that science needs a diversity of indicators that are transparent, robust, and valid, and that also allow a balanced and diverse view of success; that assessment of scientists should not blindly depend on metrics but also value human input; and that quality should be valued over quantity.

Conclusions

The objective of research assessments may be to encourage good researchers, to benefit society, or simply to advance science. Yet we show that current assessments fall short on each of these objectives. Open and transparent inter-actor dialogue is needed to understand what research assessments aim for and how they can best achieve their objective.

URL : Rethinking success, integrity, and culture in research (part 1) — a multi-actor qualitative study on success in science

DOI : https://doi.org/10.1186/s41073-020-00104-0

Survey study of research integrity officers’ perceptions of research practices associated with instances of research misconduct

Posté le par

Author : Michael Kalichman

Background

Research on research integrity has tended to focus on frequency of research misconduct and factors that might induce someone to commit research misconduct.

A definitive answer to the first question has been elusive, but it remains clear that any research misconduct is too much. Answers to the second question are so diverse, it might be productive to ask a different question: What about how research is done allows research misconduct to occur?

Methods

With that question in mind, research integrity officers (RIOs) of the 62 members of the American Association of Universities were invited to complete a brief survey about their most recent instance of a finding of research misconduct.

Respondents were asked whether one or more good practices of research (e.g., openness and transparency, keeping good research records) were present in their case of research misconduct.

Results

Twenty-four (24) of the respondents (39% response rate) indicated they had dealt with at least one finding of research misconduct and answered the survey questions. Over half of these RIOs reported that their case of research misconduct had occurred in an environment in which at least nine of the ten listed good practices of research were deficient.

Conclusions

These results are not evidence for a causal effect of poor practices, but it is arguable that committing research misconduct would be more difficult if not impossible in research environments adhering to good practices of research.

URL : Survey study of research integrity officers’ perceptions of research practices associated with instances of research misconduct

DOI : https://doi.org/10.1186/s41073-020-00103-1