Archives des mots-clés : Peer Review

Peer Review Quality and Transparency of the Peer-Review Process in Open Access and Subscription Journals

Posté le par

Background

Recent controversies highlighting substandard peer review in Open Access (OA) and traditional (subscription) journals have increased the need for authors, funders, publishers, and institutions to assure quality of peer-review in academic journals. I propose that transparency of the peer-review process may be seen as an indicator of the quality of peer-review, and develop and validate a tool enabling different stakeholders to assess transparency of the peer-review process.

Methods and Findings

Based on editorial guidelines and best practices, I developed a 14-item tool to rate transparency of the peer-review process on the basis of journals’ websites. In Study 1, a random sample of 231 authors of papers in 92 subscription journals in different fields rated transparency of the journals that published their work. Authors’ ratings of the transparency were positively associated with quality of the peer-review process but unrelated to journal’s impact factors.

In Study 2, 20 experts on OA publishing assessed the transparency of established (non-OA) journals, OA journals categorized as being published by potential predatory publishers, and journals from the Directory of Open Access Journals (DOAJ). Results show high reliability across items (α = .91) and sufficient reliability across raters. Ratings differentiated the three types of journals well.

In Study 3, academic librarians rated a random sample of 140 DOAJ journals and another 54 journals that had received a hoax paper written by Bohannon to test peer-review quality. Journals with higher transparency ratings were less likely to accept the flawed paper and showed higher impact as measured by the h5 index from Google Scholar.

Conclusions

The tool to assess transparency of the peer-review process at academic journals shows promising reliability and validity. The transparency of the peer-review process can be seen as an indicator of peer-review quality allowing the tool to be used to predict academic quality in new journals.

URL : Peer Review Quality and Transparency of the Peer-Review Process in Open Access and Subscription Journals

DOI: 10.1371/journal.pone.0147913

The Peer Reviewers’ Openness Initiative: incentivizing open research practices through peer review

Posté le par

Openness is one of the central values of science. Open scientific practices such as sharing data, materials and analysis scripts alongside published articles have many benefits, including easier replication and extension studies, increased availability of data for theory-building and meta-analysis, and increased possibility of review and collaboration even after a paper has been published. Although modern information technology makes sharing easier than ever before, uptake of open practices had been slow. We suggest this might be in part due to a social dilemma arising from misaligned incentives and propose a specific, concrete mechanism—reviewers withholding comprehensive review—to achieve the goal of creating the expectation of open practices as a matter of scientific principle.

URL : The Peer Reviewers’ Openness Initiative: incentivizing open research practices through peer review

DOI: 10.1098/rsos.150547

Peer review: The current landscape and future trends

Posté le par

This paper is based on research commissioned by the Wellcome Trust in 2015 and catalogues current initiatives and trends in the systems and processes surrounding peer review. It considers issues such as open and interactive reviews, post-publication comments and ratings, and the platforms provided by both publishers and other organisations to support such activity; third-party peer review platforms; and measures from publishers and others to provide more recognition and rewards for peer reviewers. It also speculates on likely key trends in peer review for the future.

URL : http://onlinelibrary.wiley.com/doi/10.1002/leap.1008/abstract

Peer review in megajournals compared with traditional scholarly journals: Does it make a difference?

Posté le par

A megajournal is an open-access journal that publishes any manuscript that presents scientifically trustworthy empirical results, without asking about the potential scientific contribution prior to publication. Megajournals have rapidly increased their output and are currently publishing around 50,000 articles per year.

We report on a small pilot study in which we looked at the citation distributions for articles in megajournals compared with journals with traditional peer review, which also evaluate articles for contribution and novelty.We found that elite journals with very low acceptance rates have far fewer articles with no or few citations, but that the long tail of articles with two citations or less was actually bigger in a sample of selective traditional journals in comparison with megajournals.

This indicates the need for more systematic studies, because the results raise many questions as to how efficiently the current peer review system in reality fulfils its filtering function.

URL : http://onlinelibrary.wiley.com/doi/10.1002/leap.1007/abstract

A new scholar’s perspective on open peer review

Posté le par

This article argues that the pedagogical and scholarly benefits of open peer review far outweigh those of traditional double-blind peer review, but require a shift in our perspective of the function and value of peer review – from a gate-keeping process, toward a supportive, constructive process of collaboration between peers and mentors.

URL : http://dx.doi.org/10.1080/13562517.2015.1085857

For what it’s worth – the open peer review landscape

Posté le par

Purpose

The purpose of this paper is twofold, first, to discuss the current and future issues around post-publication open peer review. Second, to highlight some of the main protagonists and platforms that encourages open peer review, pre-and post-publication.

Design/methodology/approach

The first part of the paper aims to discuss the facilitators and barriers that will enable and prevent academics engaging with the new and established platforms of scholarly communication and review. These issues are covered with the intention of proposing further dialogue within the academic community that ultimately address researchers’ concerns, whilst continuing to nurture a progressive approach to scholarly communication and review. The paper will continue to look at the prominent open post-publication platforms and tools and discuss whether in the future it will become a standard model.

Findings

 The paper identifies several problems, not exclusive to open peer review that could inhibit academics from being open with their reviews and comments of other’s research. Whilst identifies opportunities to be had by embracing a new era of academic openness.

Practical implications

 The paper summarises key platforms and arguments for open peer review and will be of interest to researchers in different disciplines as well as the wider academic community wanting to know more about scholarly communications and measurement.

Originality/value

 This paper looks at many of the new platforms that have been previously ignored and discusses issues relating to open peer review that have only been touched on in brief by other published research.

URL : http://dx.doi.org/10.1108/OIR-06-2015-0182

The quandary between communication and certification

Posté le par

Purpose

The purpose of this paper is to understand individual academics’ perception, attitudes and participation in Open Access Publishing and open scholarship and revisit some principles and designs of openness in academic publishing from the perspective of creative end-users, which helps to increase the sustainability and efficiency of open models.

Design/methodology/approach

 This paper draws on a case study of China and empirical data collected through semi-structured interviews with a wide range of academics and stakeholders.

Findings

 A separation between the communication and certification functions of publishing is identified: open initiatives are valued for efficient and interactive communication while traditional publishing still dominates the legitimacy of research publications, which leads to the quandary of individual academics operating within the transitional landscape of scholarly communication.

Practical implications

Practical recommendations for sustainable and efficient openness are derived from discussions on the difficulties associated open/social certification and the shifting maxims that govern academics from “publish or perish” to “be visible or vanish”.

Originality/value

“Openness” is defined in broad sense integrating Open Access and open scholarship to comprehensively reflect individual academics’ views in the transitional landscape of academic publishing. The research findings suggest that new open approaches are needed to address the evolving tension and conflicts between communication and certification.

URL : http://dx.doi.org/10.1108/OIR-04-2015-0129