Toward More Inclusive Metrics and Open Science to Measure Research Assessment in Earth and Natural Sciences

Authors : Olivier Pourret, Dasapta Erwin Irawan, Najmeh Shaghaei, Elenora M. van Rijsingen, Lonni Besançon

The conventional assessment of scientists relies on a set of metrics which are mostly based on the production of scientific articles and their citations. These metrics are primarily established at the journal level (e.g., the Journal Impact Factor), the article-level (e.g., times cited), and the author level (e.g., h-index; Figure 1).

These metrics form the basis of criteria that have been widely used to measure institutional reputation, as well as that of authors and research groups. By relying mostly on citations (Langfeldt et al., 2021), however, they are inherently flawed in that they provide only a limited picture of scholarly production. Indeed, citations only count document use within scholarly works and thus provide a very limited view of the use and impact of an article.

Those reveal only the superficial dimensions of a research’s impact on society. Even within academia, citations are limited since the link they express does not hold any value (Tennant et al., 2019). As an example, one could be cited for the robustness of the presented work while the other could be cited for its main limitation (Aksnes et al., 2019).

As such, two articles could be cited the same number of times for very different reasons, and relying on citations to evaluate scientific work therefore displays obvious limitations (Tahamtan et al., 2016). Beyond this issue, however, the conventional assessment of scientists is clearly beneficial to some scientists more than others and does not reflect or encourage the dissemination of knowledge back to the public that is ultimately paying scientists.

This is visible in the Earth and natural sciences which has been organized to solve local community problems in dealing with the Earth system like groundwater hazards (Irawan et al., 2021; Dwivedi et al., 2022).

URL: Toward More Inclusive Metrics and Open Science to Measure Research Assessment in Earth and Natural Sciences

DOI : https://doi.org/10.3389/frma.2022.850333

Open Up: A Survey on Open and Non-anonymized Peer Reviewing

Authors : Lonni Besançon, Niklas Rönnberg, Jonas Löwgren, Jonathan P. Tennant, Matthew Cooper

We present a discussion and analysis regarding the benefits and limitations of open and non-anonymized peer review based on literature results and responses to a survey on the reviewing process of alt.chi, a more or less open-review track within the CHI conference, the predominant conference in the field of human-computer interaction (HCI).

This track currently is the only implementation of an open-peer-review process in the field of HCI while, with the recent increase in interest in open science practices, open review is now being considered and used in other fields.

We collected 30 responses from alt.chi authors and reviewers and found that, while the benefits are quite clear and the system is generally well liked by alt.chi participants, they are reluctant to see it used in other venues.

This concurs with a number of recent studies that suggest a divergence between support for a more open review process and its practical implementation. The data and scripts are available on https://osf.io/vuw7h/, and the figures and follow-up work on http://tiny.cc/OpenReviews.

URL : Open Up: A Survey on Open and Non-anonymized Peer Reviewing

Alternative location : https://www.preprints.org/manuscript/201905.0098/v2