Champions of Transparency in Education: What Journal Reviewers Can Do to Encourage Open Science Practices

Authors : Rachel Renbarger, Jill L. Adelson, Joshua Rosenberg, Sondra M Stegenga, Olivia Lowrey, Pamela Rose Buckley, Qiyang Zhang

As the field of education and especially gifted education gradually moves towards open science, our research community increasingly values transparency and openness brought by open science practices.

Yet, individual researchers may be reluctant to adopt open science practices due to low incentives, barriers of extra workload, or lack of support to apply these in certain areas, such as qualitative research.

We encourage and give guidelines to reviewers to champion open science practices by warmly influencing authors to consider applying open science practices to quantitative, qualitative, and mixed methods research and providing ample support to produce higher-quality publications.

Instead of imposing open science practices on authors, we advocate reviewers suggest small, non-threatening, specific steps to support authors without making them feel overwhelmed, judged, or punished.

We believe that these small steps taken by reviewers will make a difference to create a more supportive environment for researchers to adopt better practices.

DOI : https://doi.org/10.35542/osf.io/xqfwb

Analysing Elsevier Journal Metadata with a New Specialized Workbench inside ICSR Lab

Authors : Ramadurai Petchiappan, Kristy James, Andrew Plume, Efthymios Tsakonas, Ana Marušić, Mario Malicki, Francisco Grimaldo, Bahar Mehmani

In this white paper we introduce Elsevier’s Peer Review Workbench which will be available via the computational platform ICSR Lab. The workbench offers a unique dataset to interested researchers who want to run research on journal evaluation and peer review processes.

We describe its properties, advantages, and limitations as well as the process of proposal application. This is a living document and will be updated on a regular basis.

DOI : https://dx.doi.org/10.2139/ssrn.4211833

Reading Peer Review : PLOS ONE and Institutional Change in Academia

Authors : Martin Paul Eve, Cameron Neylon, Daniel Paul O’Donnell, Samuel Moore, Robert Gadie, Victoria Odeniyi, Shahina Parvin

This Element describes for the first time the database of peer review reports at PLOS ONE, the largest scientific journal in the world, to which the authors had unique access.

Specifically, this Element presents the background contexts and histories of peer review, the data-handling sensitivities of this type of research, the typical properties of reports in the journal to which the authors had access, a taxonomy of the reports, and their sentiment arcs.

This unique work thereby yields a compelling and unprecedented set of insights into the evolving state of peer review in the twenty-first century, at a crucial political moment for the transformation of science.

It also, though, presents a study in radicalism and the ways in which PLOS’s vision for science can be said to have effected change in the ultra-conservative contemporary university.

URL : Reading Peer Review : PLOS ONE and Institutional Change in Academia

Original location : https://www.cambridge.org/core/elements/reading-peer-review/42F027E4C67D246DD8C3AC440A68C7A7

Attitudes, behaviours and experiences of authors of COVID-19 preprints

Authors : Narmin Rzayeva, Susana Oliveira Henriques, Stephen Pinfield, Ludo Waltman

The COVID-19 pandemic caused a rise in preprinting, apparently triggered by the need for open and rapid dissemination of research outputs. We surveyed authors of COVID-19 preprints to learn about their experience of preprinting as well as publishing in a peer-reviewed journal.

A key aim was to consider preprints in terms of their effectiveness for authors to receive feedback on their work. We also aimed to compare the impact of feedback on preprints with the impact of comments of editors and reviewers on papers submitted to journals. We observed a high rate of new adopters of preprinting who reported positive intentions regarding preprinting their future work.

This allows us to posit that the boost in preprinting may have a structural effect that will last after the pandemic. We also saw a high rate of feedback on preprints but mainly through “closed” channels – directly to the authors.

This means that preprinting was a useful way to receive feedback on research, but the value of feedback could be increased further by facilitating and promoting “open” channels for preprint feedback. At the same time, almost a quarter of the preprints that received feedback received comments resembling journal peer review.

This shows the potential of preprint feedback to provide valuable detailed comments on research. However, journal peer review resulted in a higher rate of major changes in the papers surveyed, suggesting that the journal peer review process has significant added value compared to preprint feedback.

URL : Attitudes, behaviours and experiences of authors of COVID-19 preprints

DOI : https://doi.org/10.31235/osf.io/d96yj

Accelerated Peer Review and Paper Processing Models in Academic Publishing

Authors : Jaime A. Teixeira da Silva, Yuki Yamada

Some journals and publishers offer a free or paid rapid peer review service. In the latter case, such a service is offered at a premium, i.e., for an additional fee, and authors receive, in return, a privileged service, namely faster peer review.

In the cut-throat world of survival in academia, the difference of a few weeks or months in terms of speed of peer review and publication may bring untold benefits to authors that manage to benefit from accelerated peer review.

We examine the deontological aspects behind this two-tier peer review system, including some positive, but mainly negative, aspects. Some paid accelerated peer review services thrive.

We examine the paid accelerated peer review services by Taylor & Francis, Future Medicine Ltd., Elsevier, and two stand-alone journals that are OASPA members. This suggests that there is a demand, and thus market, for faster peer review.

However, this privilege risks creating a two-tiered system that may divide academics between those who can pay versus those who cannot.

We recommend that those papers that have benefited from accelerated peer review clearly indicate this in the published papers, as either a disclaimer or within the acknowledgements, for maximum transparency of the peer review and publication process.

DOI : https://doi.org/10.1007/s12109-022-09891-4

Transparency versus anonymity: which is better to eliminate bias in peer review?

Authors: Faye Holst, Kim Eggleton, Simon Harris

Peer review is a critical component of the scientific process. When conducted properly by dedicated and competent reviewers, it helps to safeguard the quality, validity, authority and rigour of academic work. However, bias in peer review is well documented and can skew objectivity of the review and hinder fair assessment of research.

To mitigate against bias and enhance accountability, IOP Publishing has introduced two different, but complementary, approaches to all their peer-reviewed, open access (OA) journals: double-anonymous peer review and transparent peer review.

Double-anonymous peer review, where the reviewer and author identities are concealed, is designed to tackle inequality in the scholarly publishing process as it reduces bias with respect to gender, race, country of origin or affiliation.

Transparent peer review shows readers the full peer review history, including reviewer reports, editor decision letters and the authors’ responses alongside the published article. Making this process visible to the community increases accountability, allows reviewers to be recognized more for their work and can aid the training of aspiring reviewers.

IOP Publishing is the first physics publisher to adopt both of these approaches portfolio wide. In this article we discuss how applying these methods has altered different elements of the publishing process. Early indicators show that there may be a marked difference in acceptance rates across regions.

URL : Transparency versus anonymity: which is better to eliminate bias in peer review?

DOI : http://doi.org/10.1629/uksg.584

The Role of Publons in the Context of Open Peer Review

Authors : Jaime A. Teixeira da Silva, Serhii Nazarovets

Publons was a peer reviewer rewards platform that aimed to recognize the contribution that academics made during peer review to a journal. For about 10 years of its existence, Publons became the most popular service among peer reviewers.

Having gained traction and popularity, Publons was purchased in 2017 by Clarivate Analytics (now Clarivate), and many academics, journals and publishers invested time and effort to participate in Publons. Using Publons, various peer review-related experiments or pilot programs were initiated by some academic publishers regarding the introduction of open peer review into their journals’ editorial processes.

In this paper, we examine pertinent literature related to Publons, and reflect on its benefits and flaws during its short-lived history. In mid-August 2022, Clarivate fused Publons into the Web of Science platform.

Publons, as a brand peer review service, has now ceased to exist but some of the functionality remains in Web of Science while other aspects that used to be open and free at Publons are now paid-for services.

We reflect on the effect of such experiments, which initially had bold and ambitious academic objectives to fortify peer review, on academics’ trust, especially when such projects become commercialized.

DOI : https://doi.org/10.1007/s12109-022-09914-0