Archives des mots-clés : Peer Review

Peer Review Bias: A Critical Review

Posté le par

Authors : Samir Haffar, Fateh Bazerbachi, M. Hassan Murad

Various types of bias and confounding have been described in the biomedical literature that can affect a study before, during, or after the intervention has been delivered.

The peer review process can also introduce bias. A compelling ethical and moral rationale necessitates improving the peer review process. A double-blind peer review system is supported on equipoise and fair-play principles.

Triple- and quadruple-blind systems have also been described but are not commonly used. The open peer review system introduces “Skin in the Game” heuristic principles for both authors and reviewers and has a small favorable effect on the quality of published reports.

In this exposition, we present, on the basis of a comprehensive literature search of PubMed from its inception until October 20, 2017, various possible mechanisms by which the peer review process can distort research results, and we discuss the evidence supporting different strategies that may mitigate this bias.

It is time to improve the quality, transparency, and accountability of the peer review system.

DOI : https://doi.org/10.1016/j.mayocp.2018.09.004

The effect of publishing peer review reports on referee behavior in five scholarly journals

Posté le par

Authors : Giangiacomo Bravo, Francisco Grimaldo, Emilia López-Iñesta, Bahar Mehmani, Flaminio Squazzoni

To increase transparency in science, some scholarly journals are publishing peer review reports. But it is unclear how this practice affects the peer review process. Here, we examine the effect of publishing peer review reports on referee behavior in five scholarly journals involved in a pilot study at Elsevier.

By considering 9,220 submissions and 18,525 reviews from 2010 to 2017, we measured changes both before and during the pilot and found that publishing reports did not significantly compromise referees’ willingness to review, recommendations, or turn-around times.

Younger and non-academic scholars were more willing to accept to review and provided more positive and objective recommendations. Male referees tended to write more constructive reports during the pilot.

Only 8.1% of referees agreed to reveal their identity in the published report. These findings suggest that open peer review does not compromise the process, at least when referees are able to protect their anonymity.

URL : The effect of publishing peer review reports on referee behavior in five scholarly journals

DOI : https://doi.org/10.1038/s41467-018-08250-2

Peer Review of Reviewers: The Author’s Perspective

Posté le par

Authors : Ivana Drvenica, Giangiacomo Bravo, Lucija Vejmelka, Aleksandar Dekanski, Olgica Nedić

The aim of this study was to investigate the opinion of authors on the overall quality and effectiveness of reviewers’ contributions to reviewed papers. We employed an on-line survey of thirteen journals which publish articles in the field of life, social or technological sciences.

Responses received from 193 authors were analysed using a mixed-effects model in order to determine factors deemed the most important in the authors’ evaluation of the reviewers. Qualitative content analysis of the responses to open questions was performed as well.

The mixed-effects model revealed that the authors’ assessment of the competence of referees strongly depended on the final editorial decision and that the speed of the review process was influential as well.

In Ordinary Least Squares (OLS) analysis on seven questions detailing authors’ opinions, perception of review speed remained a significant predictor of the assessment. In addition, both the perceived competence and helpfulness of the reviewers significantly and positively affected the authors’ evaluation.

New models were used to re-check the value of these two factors and it was confirmed that the assessment of the competence of reviewers strongly depended on the final editorial decision.

URL : Peer Review of Reviewers: The Author’s Perspective

Alternative location : https://www.mdpi.com/2304-6775/7/1/1

Automatically Annotating Articles Towards Opening and Reusing Transparent Peer Reviews

Posté le par

Authors : Afshin Sadeghi, Sarven Capadisli, Johannes Wilm, Christoph Lange, Philipp Mayr

An increasing number of scientific publications are created in open and transparent peer review models: a submission is published first, and then reviewers are invited, or a submission is reviewed in a closed environment but then these reviews are published with the final article, or combinations of these.

Reasons for open peer review include giving better credit to reviewers and enabling readers to better appraise the quality of a publication. In most cases, the full, unstructured text of an open review is published next to the full, unstructured text of the article reviewed.

This approach prevents human readers from getting a quick impression of the quality of parts of an article, and it does not easily support secondary exploitation, e.g., for scientometrics on reviews.

While document formats have been proposed for publishing structured articles including reviews, integrated tool support for entire open peer review workflows resulting in such documents is still scarce.

We present AR-Annotator, the Automatic Article and Review Annotator which employs a semantic information model of an article and its reviews, using semantic markup and unique identifiers for all entities of interest.

The fine-grained article structure is not only exposed to authors and reviewers but also preserved in the published version. We publish articles and their reviews in a Linked Data representation and thus maximize their reusability by third-party applications.

We demonstrate this reusability by running quality-related queries against the structured representation of articles and their reviews.

URL : https://arxiv.org/abs/1812.01027

On the value of preprints: an early career researcher perspective

Posté le par

Authors : Sarvenaz Sarabipour​, Humberto J Debat, Edward Emmott, Steven Burgess, Benjamin Schwessinger, Zach Hensel

Peer-reviewed journal publication is the main means for academic researchers in the life sciences to create a permanent, public record of their work. These publications are also the de facto currency for career progress, with a strong link between journal brand recognition and perceived value.

The current peer-review process can lead to long delays between submission and publication, with cycles of rejection, revision and resubmission causing redundant peer review.

This situation creates unique challenges for early career researchers (ECRs), who rely heavily on timely publication of their work to gain recognition for their efforts. ECRs face changes in the academic landscape including the increased interdisciplinarity of life sciences research, expansion of the researcher population and consequent shifts in employer and funding demands.

The publication of preprints, publicly available scientific manuscripts posted on dedicated preprint servers prior to journal managed peer-review, can play a key role in addressing these ECR challenges.

Preprinting benefits include rapid dissemination of academic work, open access, establishing priority or concurrence, receiving feedback and facilitating collaborations. While there is a growing appreciation for and adoption of preprints, a minority of all articles in life sciences and medicine are preprinted.

The current low rate of preprint submissions in life sciences and ECR concerns regarding preprinting needs to be addressed.

We provide a perspective from an interdisciplinary group of early career researchers on the value of preprints and advocate the wide adoption of preprints to advance knowledge and facilitate career development.

URL : On the value of preprints: an early career researcher perspective

DOI : https://doi.org/10.7287/peerj.preprints.27400v1

Publish-and-Flourish: decentralized co-creation and curation of scholarly content

Posté le par

Authors : Emilija Stojmenova Duh, Andrej Duh, Uroš Droftina, Tim Kos, Urban Duh, Tanja Simonič Korošak, Dean Korošak

Scholarly communication is today immersed in publish or perish culture that propels noncooperative behaviour in the sense of strategic games played by researchers.

Here we introduce and describe a blockchain based platform for decentralized scholarly communication. The design of the platform rests on community driven publishing reviewing processes and implements incentives that promote cooperative user behaviour.

Key to achieve cooperation in blockchain based scholarly communication is to transform a static research paper into a modifiable research paper under continuous peer review process.

We describe and discuss the implementation of a modifiable research paper as a smart contract on the blockchain.

URL : https://arxiv.org/abs/1810.10263

Do funding applications where peer reviewers disagree have higher citations? A cross-sectional study

Posté le par

Authors : Adrian G Barnett, Scott R. Glisson, Stephen Gallo

Background

Decisions about which applications to fund are generally based on the mean scores of a panel of peer reviewers. As well as the mean, a large disagreement between peer reviewers may also be worth considering, as it may indicate a high-risk application with a high return.

Methods

We examined the peer reviewers’ scores for 227 funded applications submitted to the American Institute of Biological Sciences between 1999 and 2006. We examined the mean score and two measures of reviewer disagreement: the standard deviation and range.

The outcome variable was the relative citation ratio, which is the number of citations from all publications associated with the application, standardised by field and publication year.

Results

There was a clear increase in relative citations for applications with a better mean. There was no association between relative citations and either of the two measures of disagreement.

Conclusions

We found no evidence that reviewer disagreement was able to identify applications with a higher than average return. However, this is the first study to empirically examine this association, and it would be useful to examine whether reviewer disagreement is associated with research impact in other funding schemes and in larger sample sizes.

URL : Do funding applications where peer reviewers disagree have higher citations? A cross-sectional study

DOI : http://dx.doi.org/10.12688/f1000research.15479.2