Étiquette : Research Assessment

Catégories
EN

Promoting values-based assessment in review, promotion, and tenure processes

Authors : Caitlin Carter, Michael R. Dougherty, Erin C. McKiernan, Greg Tananbaum

Criteria and guidelines for review, promotion, and tenure (RPT) processes form the bedrock of institutional and departmental policies, and are a major driver of faculty behavior, influencing the time faculty spend on different activities like outreach, publishing practices, and more.

However, research shows that many RPT guidelines emphasize quantity over quality when evaluating research and teaching, and favor bibliometrics over qualitative measures of broader impact.

RPT processes rarely explicitly recognize or reward the various public dimensions of faculty work (e.g., outreach, research sharing, science communication), or, when they do, relegate them to the service category, which is undervalued and often falls heavily on women and underrepresented groups.

There is a need to correct this mismatch between institutional missions or values—often focused on aspects like community engagement, equity, diversity, and inclusion, or public good—and the behaviors that are rewarded in academic assessments. We describe recent efforts to promote RPT reform and realign institutional incentives using a values-based approach, including an overview of workshops we ran at the 2023 Council of Graduate Departments of Psychology (COGDOP) Annual Meeting, the Association for Psychological Science (APS) Annual Convention, and the American Anthropological Association (AAA) Department Leaders Summer Institute.

These workshops were designed to guide participants through the process of brainstorming what values are important to them as departments, institutions, or more broadly as disciplines, and which faculty behaviors might embody these values and could be considered in RPT evaluations. We discuss how similar activities could promote broader culture change.

URL : Promoting values-based assessment in review, promotion, and tenure processes

DOI : https://doi.org/10.21428/6ffd8432.9eadd603

Catégories
EN

The Role of Academic Libraries in Scientific Production Evaluation – the Experience of University of Zagreb, Croatia

Authors : Branka Marijanović, Tatijana Petrić, Zrinka Udiljak Bugarinovski, Višnja Novosel

Since internationally visible scientific productivity is a criterion for state evaluation of Croatian academic and scientific institutions and their scientists, Croatian academic libraries have a key role in quantitative evaluation of scientific productivity using methods such as bibliometrics, scientometrics and the like.

The aim of this case study is to identify and illustrate the current situation of library services for evaluating scientific production at the University of Zagreb, Croatia, and to make recommendations for the further development of such services, which could serve as a framework for the systematic implementation of this type of service in all libraries at the University of Zagreb and beyond.

More specifically, the purpose of this paper was to identify the existence of the bibliometric services in the libraries of the University of Zagreb (UNIZG), examine the status and involvement of university librarians in the academic advancement procedures and to identify the required competences for bibliometric experts in Croatia.

The research was conducted using the content analysis method, the survey method, and the focus group method. The research results show that although UNIZG libraries are integrated into the system of academic promotion and the role of UNIZG libraries is enshrined in Croatian regulations, the bibliometric service is not standardised at the University level.

The results also indicate that the service needs to be strengthened in terms of training of professional staff and greater investment in staff capacity and infrastructure.

The fact that the study was conducted at a single Croatian university is a possible limitation that could relate to the application of guidelines for further actions and the development of bibliometric services at national level. It would therefore be desirable to conduct future research to identify the situation at other Croatian universities as well.

It would also be necessary to determine the open science and open access policies at UNIZG through further research and, in this context, to establish guidelines for possible improvements in the processes of evaluating scientific productivity.

The results of this study make an important contribution to the possible future positioning of university libraries and UNIZG librarians in the process of evaluating scientific productivity. In addition, some practical advice is given so that this case study may be a good introductory overview for the wider academic community in relation to this topic.

URL : The Role of Academic Libraries in Scientific Production Evaluation – the Experience of University of Zagreb, Croatia

DOI : https://doi.org/10.53377/lq.13523

Catégories
EN

Viewing research assessment, the academic reward system, and academic publishing through the power/knowledge lens of Foucault

Author : Timothy D. Bowman

The academic research assessment system, the academic reward system, and the academic publishing system are interrelated mechanisms that facilitate the scholarly production of knowledge.

This article considers these systems using a Foucauldian lens to examine the power/knowledge relationships found within and through these systems. A brief description of the various systems is introduced followed by examples of instances where Foucault’s power, knowledge, discourse, and power/knowledge concepts are useful to provide a broader understanding of the norms and rules associated with each system, how these systems form a network of power relationships that reinforce and shape one another.

URL : Viewing research assessment, the academic reward system, and academic publishing through the power/knowledge lens of Foucault

DOI : https://doi.org/10.3389/frma.2023.1179376

Catégories
EN

Exploring the merits of research performance measures that comply with the San FranciscoDeclaration on Research Assessment and strategies to overcome barriers of adoption: qualitative interviews with administrators and researchers

Authors : Himani Boury, Mathieu Albert, Robert H. C. Chen, James C. L. Chow, Ralph DaCosta, Michael M. Hoffman, Behrang Keshavarz, Pia Kontos, Mary Pat McAndrews, Stephanie Protze, Anna R. Gagliardi

Background

In prior research, we identified and prioritized ten measures to assess research performance that comply with the San Francisco Declaration on Research Assessment, a principle adopted worldwide that discourages metrics-based assessment.

Given the shift away from assessment based on Journal Impact Factor, we explored potential barriers to implementing and adopting the prioritized measures.

Methods

We identified administrators and researchers across six research institutes, conducted telephone interviews with consenting participants, and used qualitative description and inductive content analysis to derive themes.

Results

We interviewed 18 participants: 6 administrators (research institute business managers and directors) and 12 researchers (7 on appointment committees) who varied by career stage (2 early, 5 mid, 5 late). Participants appreciated that the measures were similar to those currently in use, comprehensive, relevant across disciplines, and generated using a rigorous process.

They also said the reporting template was easy to understand and use. In contrast, a few administrators thought the measures were not relevant across disciplines. A few participants said it would be time-consuming and difficult to prepare narratives when reporting the measures, and several thought that it would be difficult to objectively evaluate researchers from a different discipline without considerable effort to read their work.

Strategies viewed as necessary to overcome barriers and support implementation of the measures included high-level endorsement of the measures, an official launch accompanied by a multi-pronged communication strategy, training for both researchers and evaluators, administrative support or automated reporting for researchers, guidance for evaluators, and sharing of approaches across research institutes.

Conclusions

While participants identified many strengths of the measures, they also identified a few limitations and offered corresponding strategies to address the barriers that we will apply at our organization. Ongoing work is needed to develop a framework to help evaluators translate the measures into an overall assessment.

Given little prior research that identified research assessment measures and strategies to support adoption of those measures, this research may be of interest to other organizations that assess the quality and impact of research.

URL : Exploring the merits of research performance measures that comply with the San Francisco Declaration on Research Assessment and strategies to overcome barriers of adoption: qualitative interviews with administrators and researchers

DOI : https://doi.org/10.1186/s12961-023-01001-w

Catégories
EN

The Platformisation of Scholarly Information and How to Fight It

Author : Lai Ma

The commercial control of academic publishing and research infrastructure by a few oligopolistic companies has crippled the development of open access movement and interfered with the ethical principles of information access and privacy.

In recent years, vertical integration of publishers and other service providers throughout the research cycle has led to platformisation, characterized by datafication and commodification similar to practices on social media platforms. Scholarly publications are treated as user-generated contents for data tracking and surveillance, resulting in profitable data products and services for research assessment, benchmarking and reporting.

Meanwhile, the bibliodiversity and equal open access are denied by the dominant gold open access model and the privacy of researchers is being compromised by spyware embedded in research infrastructure.

This article proposes four actions to fight the platformisation of scholarly information after a brief overview of the market of academic journals and research assessments and their implications for bibliodiversity, information access, and privacy: (1) Educate researchers about commercial publishers and APCs; (2) Allocate library budget to support scholar-led and library publishing; (3) Engage in the development of public research infrastructures and copyright reform; and (4) Advocate for research assessment reforms.

URL : The Platformisation of Scholarly Information and How to Fight It

DOI : https://doi.org/10.53377/lq.13561

Catégories
FR

« Les brevets sont à peine au rang d’une publication » : Projets de valorisation et cycle de crédibilité au CNRS

Autrice/Author : Victoria Brun

Cet article vise à expliciter la place qu’occupent les activités de valorisation dans les carrières des personnels de la recherche publique et la manière dont ils travaillent ou non à les internaliser dans le cycle de crédibilité académique (Latour & Woolgar, 1979).

À partir d’une enquête conduite dans des projets de valorisation liés au CNRS, l’analyse montre que les activités de valorisation sont pensées conjointement avec les activités académiques. Si les chercheur·se·s échouent le plus souvent à les convertir en reconnaissance sans détour par la publication, il·elle·s peuvent réinjecter cet investissement sous forme de financement et d’équipement pour d’autres travaux.

D’autres décident de les externaliser, faisant de la valorisation un à-côté de la carrière. Les doctorant·e·s et les ingénieur·e·s, qui participent pourtant à alimenter le cycle de crédibilité des chercheur·se·s, investissent des voies professionnelles parallèles. Enfin, l’engagement dans des projets de valorisation expose à des risques de décrédibilisation que les chercheur·se·s dénouent en défendant une conception du désintéressement scientifique compatible avec la perspective applicative.

La transformation de l’économie de la crédibilité se fait donc à la marge, malgré les multiples dispositifs incitatifs des institutions de recherche.

DOI : https://doi.org/10.4000/rac.30214

Catégories
EN

Contours of a research ethics and integrity perspective on open science

Authors : Tom Lindemann, Lisa Häberlein

This article argues that adopting a research ethics and integrity perspective could support researchers in operationalizing the open science guiding principle “as open as possible, as closed as necessary” in a responsible and context-sensitive manner.

To that end, the article points out why the guiding principle as such provides only a limited extent of action-guidance and outlines the practical value of ethical reflection when it comes to translating open science into responsible research practice.

The article illustrates how research ethics and integrity considerations may help researchers understand the ethical rationale underpinning open science as well as recognize that limiting openness is necessary or at least normatively permissible in some situations.

Finally, the article briefly discusses possible consequences of integrating open science into a responsibility-centered framework and implications on research assessment.

URL : Contours of a research ethics and integrity perspective on open science

DOI : https://doi.org/10.3389/frma.2023.1052353