Étiquette : Peer Review

Publié le

Open Post Publication Peer Review: An Innovation in Scientific Communication

Authors : Peiling Wang, Manasa Rath, Michael Deike, Wu Qiang

This research observes the emerging open peer review journals. In scientific publishing, transparency in peer review is a growing topic of interest for online journals. The traditional blind refereeing process has been criticized for lacking transparency.

Although the idea of open peer review (OPR) has been explored since 1980s, it is only in this decade that OPR journals are born. Towards a more open publishing model, the peer review process–once accessible only to the editors and referees—is now available to public.

The published article and its review history are being integrated into one entity; readers can submit or post comments to extend the peer process. This preliminary study observed four pioneer OPR journals representing pre-publication OPR and post-publication OPR.

Data collection focuses on publication’s lifecycle from its submission to peer approval. Preliminary results include comparisons of the level of openness and nature of interactions during refereeing process.

URL : http://trace.tennessee.edu/utk_infosciepubs/55/

Publié le

Opening Review in LIS Journals: A Status Report

Author : Emily Ford

INTRODUCTION

Peer-review practices in scholarly publishing are changing. Digital publishing mechanisms allow for open peer review, a peer review process that discloses author and reviewer identities to one another.

This model of peer review is increasingly implemented in scholarly publishing. In science, technology, engineering, and math (STEM) disciplines, open peer review is implemented in journal publishing processes, and, in the humanities and social sciences, it is often coupled with new scholarship practices, such as the digital humanities.

This article reports findings from an exploratory study on peer-review and publishing practices in Library and Information Science (LIS), focusing on LIS’s relationships with open peer review.

METHODS

Editors of LIS journals were surveyed regarding journal peer review and publishing practices.

RESULTS

This article reports the general “pulse” of attitudes and conversations regarding open peer review and discusses its challenges in LIS. Results show an ideological split between traditionally-published journals and open access and association-affiliated journals. Open access and association-affiliated journal editors are more likely to consider investigating open peer review.

DISCUSSION

The LIS community of journal editors, authors, reviewers, and readers need to discuss open peer review as well as experiment with it. Experiments with open peer review in scholarly LIS publishing will inform our praxis as librarians.

URL : Opening Review in LIS Journals: A Status Report

DOI : http://doi.org/10.7710/2162-3309.2148

Publié le

The Miracle of Peer Review and Development in Science: An Agent-Based Model

Authors : Simone Righi, Károly Takács

It is not easy to rationalize how peer review, as the current grassroots of science, can work based on voluntary contributions of reviewers. There is no rationale to write impartial and thorough evaluations.

Consequently, there is no risk in submitting low-quality work by authors. As a result, scientists face a social dilemma: if everyone acts according to his or her own self-interest, low scientific quality is produced. Still, in practice, reviewers as well as authors invest high effort in reviews and submissions.

We examine how the increased relevance of public good benefits (journal impact factor), the editorial policy of handling incoming reviews, and the acceptance decisions that take into account reputational information can help the evolution of high-quality contributions from authors.

High effort from the side of reviewers is problematic even if authors cooperate: reviewers are still best off by producing low-quality reviews, which does not hinder scientific development, just adds random noise and unnecessary costs to it.

We show with agent-based simulations that tacit agreements between authors that are based on reciprocity might decrease these costs, but does not result in superior scientific quality. Our study underlines why certain self-emerged current practices, such as the increased importance of journal metrics, the reputation-based selection of reviewers, and the reputation bias in acceptance work efficiently for scientific development.

Our results find no answers, however, how the system of peer review with impartial and thorough evaluations could be sustainable jointly with rapid scientific development.

URL : http://arxiv.org/abs/1607.02574

Publié le

The effects of an editor serving as one of the reviewers during the peer-review process

Authors : Marco Giordan, Attila Csikasz-Nagy, Andrew M. Collings

Background

Publishing in scientific journals is one of the most important ways in which scientists disseminate research to their peers and to the wider public.

Pre-publication peer review underpins this process, but peer review is subject to various criticisms and is under pressure from growth in the number of scientific publications.

Methods

Here we examine an element of the editorial process at eLife, in which the Reviewing Editor usually serves as one of the referees, to see what effect this has on decision times, decision type, and the number of citations.

We analysed a dataset of 8,905 research submissions to eLife since June 2012, of which 2,750 were sent for peer review, using R and Python to perform the statistical analysis.

Results

The Reviewing Editor serving as one of the peer reviewers results in faster decision times on average, with the time to final decision ten days faster for accepted submissions (n=1,405) and 5 days faster for papers that were rejected after peer review (n=1,099).

There was no effect on whether submissions were accepted or rejected, and a very small (but significant) effect on citation rates for published articles where the Reviewing Editor served as one of the peer reviewers.

Conclusions

An important aspect of eLife’s peer-review process is shown to be effective, given that decision times are faster when the Reviewing Editor serves as a reviewer. Other journals hoping to improve decision times could consider adopting a similar approach.

URL : The effects of an editor serving as one of the reviewers during the peer-review process

DOI : http://dx.doi.org/10.12688/f1000research.8452.1

Publié le

Open peer review : from an experiment to a model. A narrative of an open peer review experimentation

This article narrates the development of the experimentation of an open peer review and open commentary protocols. This experiment concerns propositions of articles for the environmental sciences journal VertigO, digital and open access scientific publication.

This experiment did not last long enough (4 months) and was not deployed on a large enough corpus (10 preprints) to lead to firm quantitative conclusions. However, it highlights practical leads and thoughts about the potentialities and the limitations of the open review processes – in the broadest sense – for scientific publishing.

Based on the exemplary of the experiment and a participant observation as a copy-editor devoted to open peer review, the article finally proposes a model from the experimented prototype.

This model, named OPRISM, could be implemented on other publishing contexts for social sciences and humanities. Central and much debated activity in the academic world, peer review refers to different practices such as control, validation, allocation and contradiction exercised by the scientific community for itself.

Its scope is wide: from the allocation for funding to the relevance of a recruitment. According to common sense, the control of the scientific community by itself is a guarantee of scientific quality.

This issue became even more important in an international context of competition between universities and between scholars themselves.

URL : Open peer review : from an experiment to a model

Alternative location : https://hal.archives-ouvertes.fr/hal-01302597

Publié le

Towards a (De)centralization-Based Typology of Peer Production

Online peer-production platforms facilitate the coordination of creative work and services. Generally considered as empowering participatory tools and a source of common good, they can also be, however, alienating instruments of digital labour.

This paper proposes a typology of peer-production platforms, based on the centralization/decentralization levels of several of their design features. Between commons-based peer-production and crowdsourced, user-generated content “enclosed” by corporations, a wide range of models combine different social, political, technical and economic arrangements.

This combined analysis of the level of (de)centralization of platform features provides information on emancipation capabilities in a more granular way than a market-based qualification of platforms, based on the nature of ownership or business models only.

The five selected features of the proposed typology are: ownership of means of production, technical architecture/design, social organization/governance of work patterns, ownership of the peer-produced resource, and value of the output.

URL : Towards a (De)centralization-Based Typology of Peer Production

Alternative location : http://triplec.at/index.php/tripleC/article/view/728

Publié le

Évaluation ouverte par les pairs : de l’expérimentation à la modélisation : Récit d’une expérience d’évaluation ouverte par les pairs

Cet article relate le déroulement de l’expérimentation d’un dispositif d’évaluation ouverte par les pairs et de commentaire ouvert, pour des propositions d’articles à la revue en sciences de l’environnement VertigO, publication scientifique électronique en accès libre.

Si cette expérimentation ne s’est pas déroulée sur un temps assez long (4 mois) et un corpus assez étendu (10 manuscrits) pour en tirer des conclusions quantitatives fermes, elle expose néanmoins des pistes et des réflexions concrètes sur les potentialités et les limites de l’ouverture des processus d’évaluation – au sens large – pour la publication scientifique.

Se basant sur l’exemplarité de l’expérience et une observation participante en tant que secrétaire de rédaction consacré à l’évaluation ouverte, l’article propose finalement la modélisation du prototype expérimenté. Ce modèle, surnommé OPRISM, pourrait être utilisé dans d’autres cadres éditoriaux pour les sciences humaines et sociales.

URL : https://hal-paris1.archives-ouvertes.fr/hal-01283582v1