Archives des mots-clés : Peer Review

A multi-disciplinary perspective on emergent and future innovations in peer review

Posté le par

Authors : Jonathan P. Tennant, Jonathan M. Dugan, Daniel Graziotin, Damien C. Jacques, François Waldner, Daniel Mietchen, Yehia Elkhatib, Lauren B. Collister, Christina K. Pikas, Tom Crick, Paola Masuzzo, Anthony Caravaggi, Devin R. Berg, Kyle E. Niemeyer, Tony Ross-Hellauer, Sara Mannheimer, Lillian Rigling, Daniel S. Kat, Bastian Greshake Tzovaras, Josmel Pacheco-Mendoza, Nazeefa Fatima, Marta Poblet, Marios Isaakidis, Dasapta Erwin Irawan, Sébastien Renaut, Christopher R. Madan, Lisa Matthias, Jesper Nørgaard Kjær, Daniel Paul O’Donnell, Cameron Neylon, Sarah Kearns, Manojkumar Selvaraju, Julien Colomb

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure?

Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research.

With the advent of Web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review.

We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation.

Ideally, any new systems will demonstrate that they out-perform current models while avoiding as many of the biases of existing systems as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages.

We also propose a novel hybrid platform model that, at least partially, resolves many of the technical and social issues associated with peer review, and can potentially disrupt the entire scholarly communication system.

Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

URL : A multi-disciplinary perspective on emergent and future innovations in peer review

DOI : http://dx.doi.org/10.12688/f1000research.12037.1

 

A Proposed Currency System for Academic Peer Review Payments Using the BlockChain Technology

Posté le par

Author : Michael Spearpoint

Peer review of scholarly papers is seen to be a critical step in the publication of high quality outputs in reputable journals.

However, it appears that there are few incentives for researchers to agree to conduct suitable reviews in a timely fashion and in some cases unscrupulous practices are occurring as part of the production of academic research output. Innovations in internet-based technologies mean that there are ways in which some of the challenges can be addressed.

In particular, this paper proposes a new currency system using the BlockChain as its basis that provides a number of solutions. Potential benefits and problems of using the technology are discussed in the paper and these will need further investigation should the idea develop further.

Ultimately, the currency could be used as an alternative publication metric for authors, institutions and journals.

URL : A Proposed Currency System for Academic Peer Review Payments Using the BlockChain Technology

Alternative location : http://www.mdpi.com/2304-6775/5/3/19

The false academy: predatory publishing in science and bioethics

Posté le par

Authors : Stefan Eriksson, Gert Helgesson

This paper describes and discusses the phenomenon ‘predatory publishing’, in relation to both academic journals and books, and suggests a list of characteristics by which to identify predatory journals. It also raises the question whether traditional publishing houses have accompanied rogue publishers upon this path.

It is noted that bioethics as a discipline does not stand unaffected by this trend. Towards the end of the paper it is discussed what can and should be done to eliminate or reduce the effects of this development.

The paper concludes that predatory publishing is a growing phenomenon that has the potential to greatly affect both bioethics and science at large.

Publishing papers and books for profit, without any genuine concern for content, but with the pretence of applying authentic academic procedures of critical scrutiny, brings about a worrying erosion of trust in scientific publishing.

URL : The false academy: predatory publishing in science and bioethics

DOI :10.1007/s11019-016-9740-3

Global Data Quality Assessment and the Situated Nature of “Best” Research Practices in Biology

Posté le par

Author : Sabina Leonelli

This paper reflects on the relation between international debates around data quality assessment and the diversity characterising research practices, goals and environments within the life sciences.

Since the emergence of molecular approaches, many biologists have focused their research, and related methods and instruments for data production, on the study of genes and genomes.

While this trend is now shifting, prominent institutions and companies with stakes in molecular biology continue to set standards for what counts as ‘good science’ worldwide, resulting in the use of specific data production technologies as proxy for assessing data quality.

This is problematic considering (1) the variability in research cultures, goals and the very characteristics of biological systems, which can give rise to countless different approaches to knowledge production; and (2) the existence of research environments that produce high-quality, significant datasets despite not availing themselves of the latest technologies.

Ethnographic research carried out in such environments evidences a widespread fear among researchers that providing extensive information about their experimental set-up will affect the perceived quality of their data, making their findings vulnerable to criticisms by better-resourced peers. T

hese fears can make scientists resistant to sharing data or describing their provenance. To counter this, debates around Open Data need to include critical reflection on how data quality is evaluated, and the extent to which that evaluation requires a localised assessment of the needs, means and goals of each research environment.

URL : Global Data Quality Assessment and the Situated Nature of “Best” Research Practices in Biology

DOI : http://doi.org/10.5334/dsj-2017-032

Decentralized creation of academic documents using a Network Attached Storage (NAS) server

Posté le par

Authors : Johannes Wilm, Afshin Sadeghi, Christoph Lange, Philipp Mayr

Scholarly document creation continues to face various obstacles. Scholarly text production requires more complex word processors than other forms of texts because of the complex structures of citations, formulas and figures.

The need for peer review, often single-blind or double-blind, creates needs for document management that other texts do not require. Additionally, the need for collaborative editing, security and strict document access rules means that many existing word processors are imperfect solutions for academics.

Nevertheless, most papers continue to be written using Microsoft Word (Sadeghi et al. 2017).

We here analyze some of the problems with existing academic solutions and then present an argument why we believe that running an open source academic writing solution for academic purposes, such as Fidus Writer, on a Network Attached Storage (NAS) server could be a viable alternative.

URL : https://arxiv.org/abs/1706.04185

Peer reviewing: a private affair between the individual researcher and the publishing houses, or a responsibility of the university?

Posté le par

Authors : Leif Longva, Eirik Reierth, Lars Moksness, Bård Smedsrød

Peer reviewing is mandatory for scientific journals as quality control of submitted manuscripts, for universities to rank applicants for scientific positions, and for funding agencies to rank grant applications.

In spite of this deep dependency of peer reviewing throughout the entire academic realm, universities exhibit a peculiar lack of interest in this activity.

The aim of this article is to show that by taking an active interest in peer reviewing the universities will take control over the management and policy shaping of scientific publishing, a regime that is presently largely controlled by the big publishing houses.

The benefits of gaining control of scientific publishing policy include the possibility to implement open access publishing and to reduce the unjustifiably high subscription rates currently charged by some of the major publishing houses.

A common international clean-up action is needed to move this pivotal element of scientific publishing from the dark hiding places of the scientific journals to where it should be managed: namely, at the universities.

In addition to the economic benefits, we postulate that placing peer reviewing at the universities will improve the quality of published research.

DOI : http://dx.doi.org/10.3998/3336451.0020.103

TrueReview: A Platform for Post-Publication Peer Review

Posté le par

Authors : Luca de Alfaro, Marco Faella

In post-publication peer review, scientific contributions are first published in open-access forums, such as arXiv or other digital libraries, and are subsequently reviewed and possibly ranked and/or evaluated.

Compared to the classical process of scientific publishing, in which review precedes publication, post-publication peer review leads to faster dissemination of ideas, and publicly-available reviews. The chief concern in post-publication reviewing consists in eliciting high-quality, insightful reviews from participants.

We describe the mathematical foundations and structure of TrueReview, an open-source tool we propose to build in support of post-publication review.

In TrueReview, the motivation to review is provided via an incentive system that promotes reviews and evaluations that are both truthful (they turn out to be correct in the long run) and informative (they provide significant new information).

TrueReview organizes papers in venues, allowing different scientific communities to set their own submission and review policies. These venues can be manually set-up, or they can correspond to categories in well-known repositories such as arXiv.

The review incentives can be used to form a reviewer ranking that can be prominently displayed alongside papers in the various disciplines, thus offering a concrete benefit to reviewers. The paper evaluations, in turn, reward the authors of the most significant papers, both via an explicit paper ranking, and via increased visibility in search.

URL : https://arxiv.org/abs/1608.07878