Archives des mots-clés : Peer Review

Does the use of open, non-anonymous peer review in scholarly publishing introduce bias? Evidence from the F1000 post-publication open peer review publishing model

Posté le par

Authors : Mike Thelwall, Verena Weigert, Liz Allen, Zena Nyakoojo, Eleanor-Rose Papas

This study examines whether there is any evidence of bias in two areas of common critique of open, non-anonymous peer review – and used in the post-publication, peer review system operated by the open-access scholarly publishing platform F1000Research.

First, is there evidence of bias where a reviewer based in a specific country assesses the work of an author also based in the same country? Second, are reviewers influenced by being able to see the comments and know the origins of previous reviewer?

Scrutinising the open peer review comments published on F1000Research, we assess the extent of two frequently cited potential influences on reviewers that may be the result of the transparency offered by a fully attributable, open peer review publishing model: the national affiliations of authors and reviewers, and the ability of reviewers to view previously-published reviewer reports before submitting their own.

The effects of these potential influences were investigated for all first versions of articles published by 8 July 2019 to F1000Research. In 16 out of the 20 countries with the most articles, there was a tendency for reviewers based in the same country to give a more positive review.

The difference was statistically significant in one. Only 3 countries had the reverse tendency. Second, there is no evidence of a conformity bias. When reviewers mentioned a previous review in their peer review report, they were not more likely to give the same overall judgement.

Although reviewers who had longer to potentially read a previously published reviewer reports were slightly less likely to agree with previous reviewer judgements, this could be due to these articles being difficult to judge rather than deliberate non-conformity.

URL : https://arxiv.org/abs/1911.03379

Which Are the Tools Available for Scholars? A Review of Assisting Software for Authors during Peer Reviewing Process

Posté le par

Authors : J. Israel Martínez-López, Samantha Barrón-González, Alejandro Martínez López

There is a large amount of Information Technology and Communication (ITC) tools that surround scholar activity. The prominent place of the peer-review process upon publication has promoted a crowded market of technological tools in several formats.

Despite this abundance, many tools are unexploited or underused because they are not known by the academic community. In this study, we explored the availability and characteristics of the assisting tools for the peer-reviewing process.

The aim was to provide a more comprehensive understanding of the tools available at this time, and to hint at new trends for further developments. The result of an examination of literature assisted the creation of a novel taxonomy of types of software available in the market.

This new classification is divided into nine categories as follows: (I) Identification and social media, (II) Academic search engines, (III) Journal-abstract matchmakers, (IV) Collaborative text editors, (V) Data visualization and analysis tools, (VI) Reference management, (VII) Proofreading and plagiarism detection, (VIII) Data archiving, and (IX) Scientometrics and Altmetrics.

Considering these categories and their defining traits, a curated list of 220 software tools was completed using a crowdfunded database (AlternativeTo) to identify relevant programs and ongoing trends and perspectives of tools developed and used by scholars.

URL : Which Are the Tools Available for Scholars? A Review of Assisting Software for Authors during Peer Reviewing Process

DOI : https://doi.org/10.3390/publications7030059

 

Bye Bye Peer-Reviewed Publishing

Posté le par

Authors : Miguel Abambres, Tony Salloom, Nejra Beganovic, Rafal Dojka, SergioRoncallo-Dow

This work is the continuation of a ‘revolution’ started with “Research Counts, Not the Journal”. Own and published opinions from worldwide scientists on critical issues of peer-reviewed publishing are presented.

In my opinion, peer-reviewed publishing is a quite flawed process (in many way) that has greatly harmed Science for a long time – it has been imposed by most academic and science funding institutions as the only way to assess scientific performance.

Unfortunately, most academics still follow that path, even though I believe most do it for the fear of losing their job or not being promoted. This paper aims to encourage (i) a full disruption of peer-reviewed publishing and (ii) the use of free eprint repositories for a sustainable academic/scientific publishing, i.e. healthier (no stress/distress associated to the peer review stage and the long waiting for publication) and more economic, effective and efficient (research is made immediately available and trackable/citable to anyone).

On the other hand, it should be pointed out that nothing exists against scientific publishers/journals – actually it´s perfectly normal that any company wants to implement its own quality criteria.

This paper is just the way chosen to promote the quick implementation of suitable policies for research evaluation.

URL : Bye Bye Peer-Reviewed Publishing

Alternative location : https://hal.archives-ouvertes.fr/hal-02114531v4

The limitations to our understanding of peer review

Posté le par

Authors : Jonathan P. Tennant, Tony Ross-Hellauer

Peer review is embedded in the core of our scholarly knowledge generation systems, conferring legitimacy on research while distributing academic capital and prestige on individuals.

Despite its critical importance, it curiously remains poorly understood in a number of dimensions. In order to address this, we have programmatically analysed peer review to assess where the major gaps in our theoretical and empirical understanding of it lie.

We distill this into core themes around editorial accountability, the subjectivity and bias of reviewers, the function and quality of peer review, the role of the reviewer, the social and epistemic implications of peer review, and regarding innovations in open peer review platforms and services.

We use this to present a guide for the future of peer review, and the development of a new research discipline based on the study of peer review. Such a field requires sustained funding and commitment from publishers and research funders, who both have a commitment to uphold the integrity of the published scholarly record.

This will require the design of a consensus for a minimal set of standards for what constitutes peer review, and the development of a shared data infrastructure to support this.

We recognise that many of the criticisms attributed to peer review might reflect wider issues within academia and wider society, and future care will be required in order to carefully demarcate and address these.

DOI : https://doi.org/10.31235/osf.io/jq623

Open Up: A Survey on Open and Non-anonymized Peer Reviewing

Posté le par

Authors : Lonni Besançon, Niklas Rönnberg, Jonas Löwgren, Jonathan P. Tennant, Matthew Cooper

We present a discussion and analysis regarding the benefits and limitations of open and non-anonymized peer review based on literature results and responses to a survey on the reviewing process of alt.chi, a more or less open-review track within the CHI conference, the predominant conference in the field of human-computer interaction (HCI).

This track currently is the only implementation of an open-peer-review process in the field of HCI while, with the recent increase in interest in open science practices, open review is now being considered and used in other fields.

We collected 30 responses from alt.chi authors and reviewers and found that, while the benefits are quite clear and the system is generally well liked by alt.chi participants, they are reluctant to see it used in other venues.

This concurs with a number of recent studies that suggest a divergence between support for a more open review process and its practical implementation. The data and scripts are available on https://osf.io/vuw7h/, and the figures and follow-up work on http://tiny.cc/OpenReviews.

URL : Open Up: A Survey on Open and Non-anonymized Peer Reviewing

Alternative location : https://www.preprints.org/manuscript/201905.0098/v2

 

Acceptance rates of scholarly peer-reviewed journals: A literature survey

Posté le par

Author : Bo-Christer Bjórk

The acceptance rate of scholarly journals is an important selection criterion for authors choosing where to submit their manuscripts. Unfortunately, information about the acceptance (or rejection rates) of individual journals is seldom available.

This article surveys available systematic information and studies of acceptance rates. The overall global average is around 35-40%. There are significant differences between fields of science, with biomedicine having higher acceptance rates compared to for instance the social sciences.

Open access journals usually have higher acceptance rates than subscription journals, and this is particularly true for so-called OA mega-journals, which have peer review criteria focusing on sound science only.

URL : https://recyt.fecyt.es/index.php/EPI/article/view/epi.2019.jul.07

The F3-index. Valuing reviewers for scholarly journals

Posté le par

Authors : Federico Bianchi, Francisco Grimaldo, Flaminio Squazzoni

This paper presents an index that measures reviewer contribution to editorial processes of scholarly journals. Following a metaphor of ranking algorithms in sports tournaments, we created an index that considers reviewers on different context-specific dimensions, i.e., report delivery time, the length of the report and the alignment of recommendations to editorial decisions.

To test the index, we used a dataset of peer review in a multi-disciplinary journal, including 544 reviewers on 606 submissions in six years. Although limited by sample size, the test showed that the index identifies outstanding contributors and weak performing reviewers efficiently.

Our index is flexible, contemplates extensions and could be incorporated into available scholarly journal management tools. It can assist editors in rewarding high performing reviewers and managing editorial turnover.

URL : The F3-index. Valuing reviewers for scholarly journals

DOI : https://doi.org/10.1016/j.joi.2018.11.007