Roles and Responsibilities for Peer Reviewers of International Journals

Author : Carol Nash

There is a noticeable paucity of recently published research on the roles and responsibilities of peer reviewers for international journals. Concurrently, the pool of these peer reviewers is decreasing. Using a narrative research method developed by the author, this study questioned these roles and responsibilities through the author’s assessment in reviewing for five publishing houses July–December 2022, in comparison with two recent studies regarding peer review, and the guidelines of the five publishing houses.

What should be most important in peer review is found discrepant among the author, those judging peer review in these publications, and the five publishing houses. Furthermore, efforts to increase the pool of peer reviewers are identified as ineffective because they focus on the reviewer qua reviewer, rather than on their primary role as researchers.

To improve consistency, authors have regularly called for peer review training. Yet, this advice neglects to recognize the efforts of journals in making their particular requirements for peer review clear, comprehensive and readily accessible.

Consequently, rather than peer reviewers being trained and rewarded as peer reviewers, journals are advised to make peer review a requirement for research publication, and their guidelines necessary reading and advice to follow for peer reviewers.

URL : Roles and Responsibilities for Peer Reviewers of International Journals

DOI : https://doi.org/10.3390/publications11020032

Metrics and peer review agreement at the institutional level

Authors : Vincent A Traag, Marco Malgarini, Sarlo Scipione

In the past decades, many countries have started to fund academic institutions based on the evaluation of their scientific performance. In this context, post-publication peer review is often used to assess scientific performance. Bibliometric indicators have been suggested as an alternative to peer review.

A recurrent question in this context is whether peer review and metrics tend to yield similar outcomes. In this paper, we study the agreement between bibliometric indicators and peer review based on a sample of publications submitted for evaluation to the national Italian research assessment exercise (2011–2014).

In particular, we study the agreement between bibliometric indicators and peer review at a higher aggregation level, namely the institutional level. Additionally, we also quantify the internal agreement of peer review at the institutional level. We base our analysis on a hierarchical Bayesian model using cross-validation.

We find that the level of agreement is generally higher at the institutional level than at the publication level. Overall, the agreement between metrics and peer review is on par with the internal agreement among two reviewers for certain fields of science in this particular context.

This suggests that for some fields, bibliometric indicators may possibly be considered as an alternative to peer review for the Italian national research assessment exercise. Although results do not necessarily generalise to other contexts, it does raise the question whether similar findings would obtain for other research assessment exercises, such as in the United Kingdom.

URL : https://arxiv.org/abs/2006.14830

Investigation of potential gender bias in the peer review system at Reproduction

Authors : Marie BiolkováTom MooreKaren SchindlerKarl SwannAndy VailLindsay FlookHelen DickGreg FitzharrisChristopher A. PriceNorah Spears

This study examined whether publication outcome was affected by the gender of author, handling associate editor (AE), or reviewer, and whether there was gender bias in reviewer selection, in the journal Reproduction.

Analyses were carried out on 4289 original research manuscripts submitted to the journal between 2007 and 2019. Both female and male AEs appointed more male reviewers than female reviewers, but female AEs were significantly more likely to appoint female reviewers than male AEs were (p < 0.001).

When examining the gender of either first or last author manuscripts, those with female authors that were reviewed by female reviewers received better scores than those with male authors that were reviewed by female reviewers (p < 0.05): where the reviewer was male, no such effect was observed.

Acceptance rates of manuscripts were similar for both female and male authors, whether first or last, regardless of AE gender. Overall, there was no significant correlation between gender of first or last author, or of AE, on the likelihood of acceptance of a research paper.

These data suggest no bias against female authors during the peer review process in this reproductive biology journal.

URL : Investigation of potential gender bias in the peer review system at Reproduction

DOI : https://doi.org/10.1002/leap.1537

Scholarly publishing and peer review in the Global South: the role of the reviewer

Author : Peter Lor

Peer review is an integral part of contemporary scholarly publishing, especially journal publishing. Work submitted by scholars from all parts of the world is subjected to it. This includes submissions by scholars from the Global South, who wish to publish in “international” journals or in local journals which follow the same model.

These authors may not be native English speakers and may be unfamiliar with the conventions of Western scholarship. Many of them conduct research and write their manuscripts under challenging circumstances.

They may find it difficult to comply with the requirements of the journals to which they submit their articles. Their manuscripts quite often pose challenges to the peer reviewers.

The purpose of this article is to provide some background on scholarly publishing in the Global South and the challenges those colleagues face, and to outline what this may mean for the role of the reviewer.

URL : Scholarly publishing and peer review in the Global South: the role of the reviewer

DOI : https://doi.org/10.36253/jlis.it-512

Global Trends in Knowledge Production and the Evolving Peer Review Process

Author : Steven Witt

This essay thus seeks to provide further critique and clarity to the peer review process and the ways in which management of peer review is evolving. These changes occur within a context of massive growth in the knowledge production process: global trends, information technologies, and policies that encourage more people globally to take part in the research process.

Associated with these global changes are stressors on the peer review process and particularly questions about who gets to be a peer reviewer and who has the right to produce knowledge under these processes.

Less a formal review and analysis of peer review across LIS, this essay takes the form of an autoethnographic narrative that that seeks to draw upon the researcher’s personal observations, experience, and reflections to critically examine changes to the peer review system that are taking place.

URL : Global Trends in Knowledge Production and the Evolving Peer Review Process

DOI : https://doi.org/10.36253/jlis.it-515

Champions of Transparency in Education: What Journal Reviewers Can Do to Encourage Open Science Practices

Authors : Rachel Renbarger, Jill L. Adelson, Joshua Rosenberg, Sondra M Stegenga, Olivia Lowrey, Pamela Rose Buckley, Qiyang Zhang

As the field of education and especially gifted education gradually moves towards open science, our research community increasingly values transparency and openness brought by open science practices.

Yet, individual researchers may be reluctant to adopt open science practices due to low incentives, barriers of extra workload, or lack of support to apply these in certain areas, such as qualitative research.

We encourage and give guidelines to reviewers to champion open science practices by warmly influencing authors to consider applying open science practices to quantitative, qualitative, and mixed methods research and providing ample support to produce higher-quality publications.

Instead of imposing open science practices on authors, we advocate reviewers suggest small, non-threatening, specific steps to support authors without making them feel overwhelmed, judged, or punished.

We believe that these small steps taken by reviewers will make a difference to create a more supportive environment for researchers to adopt better practices.

DOI : https://doi.org/10.35542/osf.io/xqfwb

Analysing Elsevier Journal Metadata with a New Specialized Workbench inside ICSR Lab

Authors : Ramadurai Petchiappan, Kristy James, Andrew Plume, Efthymios Tsakonas, Ana Marušić, Mario Malicki, Francisco Grimaldo, Bahar Mehmani

In this white paper we introduce Elsevier’s Peer Review Workbench which will be available via the computational platform ICSR Lab. The workbench offers a unique dataset to interested researchers who want to run research on journal evaluation and peer review processes.

We describe its properties, advantages, and limitations as well as the process of proposal application. This is a living document and will be updated on a regular basis.

DOI : https://dx.doi.org/10.2139/ssrn.4211833