Archives des mots-clés : Peer Review

Retrospective analysis of the quality of reports by author-suggested and non-author-suggested reviewers in journals operating on open or single-blind peer review models

Statut

Objectives

To assess whether reports from reviewers recommended by authors show a bias in quality and recommendation for editorial decision, compared with reviewers suggested by other parties, and whether reviewer reports for journals operating on open or single-blind peer review models differ with regard to report quality and reviewer recommendations.

Design

Retrospective analysis of the quality of reviewer reports using an established Review Quality Instrument, and analysis of reviewer recommendations and author satisfaction surveys.

Setting

BioMed Central biology and medical journals. BMC Infectious Diseases and BMC Microbiology are similar in size, rejection rates, impact factors and editorial processes, but the former uses open peer review while the latter uses single-blind peer review. The Journal of Inflammation has operated under both peer review models.

Sample

Two hundred reviewer reports submitted to BMC Infectious Diseases, 200 reviewer reports submitted to BMC Microbiology and 400 reviewer reports submitted to the Journal of Inflammation.

Results

For each journal, author-suggested reviewers provided reports of comparable quality to non-author-suggested reviewers, but were significantly more likely to recommend acceptance, irrespective of the peer review model (p<0.0001 for BMC Infectious Diseases, BMC Microbiology and the Journal of Inflammation). For BMC Infectious Diseases, the overall quality of reviewer reports measured by the Review Quality Instrument was 5% higher than for BMC Microbiology (p=0.042). For the Journal of Inflammation, the quality of reports was the same irrespective of the peer review model used.

Conclusions

Reviewers suggested by authors provide reports of comparable quality to non-author-suggested reviewers, but are significantly more likely to recommend acceptance. Open peer review reports for BMC Infectious Diseases were of higher quality than single-blind reports for BMC Microbiology. There was no difference in quality of peer review in the Journal of Inflammation under open peer review compared with single blind.

URL : Retrospective analysis of the quality of reports by author-suggested and non-author-suggested reviewers in journals operating on open or single-blind peer review models

DOI : 10.1136/bmjopen-2015-008707

Review times in peer review: quantitative analysis of editorial workflows

Statut

We examine selected aspects of peer review and suggest possible improvements. To this end, we analyse a dataset containing information about 300 papers submitted to the Biochemistry and Biotechnology section of the Journal of the Serbian Chemical Society. After separating the peer review process into stages that each review has to go through, we use a weighted directed graph to describe it in a probabilistic manner and test the impact of some modifications of the editorial policy on the efficiency of the whole process.

URL : http://arxiv.org/abs/1508.01134

From Peer-Reviewed to Peer-Reproduced in Scholarly Publishing: The Complementary Roles of Data Models and Workflows in Bioinformatics

Posté le par

Motivation

Reproducing the results from a scientific paper can be challenging due to the absence of data and the computational tools required for their analysis. In addition, details relating to the procedures used to obtain the published results can be difficult to discern due to the use of natural language when reporting how experiments have been performed.

The Investigation/Study/Assay (ISA), Nanopublications (NP), and Research Objects (RO) models are conceptual data modelling frameworks that can structure such information from scientific papers. Computational workflow platforms can also be used to reproduce analyses of data in a principled manner. We assessed the extent by which ISA, NP, and RO models, together with the Galaxy workflow system, can capture the experimental processes and reproduce the findings of a previously published paper reporting on the development of SOAPdenovo2, a de novo genome assembler.

Results

Executable workflows were developed using Galaxy, which reproduced results that were consistent with the published findings. A structured representation of the information in the SOAPdenovo2 paper was produced by combining the use of ISA, NP, and RO models. By structuring the information in the published paper using these data and scientific workflow modelling frameworks, it was possible to explicitly declare elements of experimental design, variables, and findings.

The models served as guides in the curation of scientific information and this led to the identification of inconsistencies in the original published paper, thereby allowing its authors to publish corrections in the form of an errata.”

URL : From Peer-Reviewed to Peer-Reproduced in Scholarly Publishing: The Complementary Roles of Data Models and Workflows in Bioinformatics

DOI : 10.1371/journal.pone.0127612

Hybrid Review: Taking SoTL Beyond Traditional Peer Review for Journal Publication

Statut

“Developments in emergent technology offer innovative solutions for facilitating a hybrid review process; we examine a unique combination of private–peer and open–public review uniquely relevant for disseminating the scholarship of teaching and learning (SoTL) through development of the Journal of Instructional Research (JIR). An analysis of the hybrid review process (combining the strengths of traditional peer review with an integrative public review process) revealed substantial reviewer participation that contributed to a well–rounded, engaged review process. Public review feedback constructively addressed the value and relevance of the implications, methodology, content and written quality of the manuscripts; an additional layer of private, peer review further refined the manuscripts to determine suitability for publication. This, in turn, created a space where refinement of content, structure, and design of SoTL research was achieved through an interactive process of scholarly inquiry and dialogue.”

DOI: http://dx.doi.org/10.3998/3336451.0018.202

Peer review: still king in the digital age

Statut

“The article presents one of the main findings of an international study of 4,000 academic researchers that examined how trustworthiness is determined in the digital environment when it comes to scholarly reading, citing, and publishing. The study shows that peer review is still the most trustworthy characteristic of all. There is, though, a common perception that open access journals are not peer reviewed or do not have proper peer-review systems. Researchers appear to have moved inexorably from a print-based system to a digital system, but it has not significantly changed the way they decide what to trust. They do not trust social media. Only a minority – although significantly mostly young and early career researchers – thought that social media are anything other than more appropriate to personal interactions and peripheral to their professional/academic lives. There are other significant differences, according to the age of the researcher. Thus, in regard to choosing an outlet for publication of their work, young researchers are much less concerned with the fact that it is peer reviewed.”

URL : http://ciber-research.eu/download/20140120-Peer_review-Learned_Publishing_2015.pdf

Data without Peer: Examples of Data Peer Review in the Earth Sciences

Statut

“Peer review of data is an important process if data is to take its place as a first class research output. Much has been written about the theoretical aspects of peer review, but not as much about the actual process of doing it. This paper takes an experimental view, and selects seven datasets, all from the Earth Sciences and with DOIs from DataCite, and attempts to review them, with varying levels of success. Key issues identified from these case studies include the necessity of human readable metadata, accessibility of datasets, and permanence of links to and accessibility of metadata stored in other locations.”

URL : http://www.dlib.org/dlib/january15/callaghan/01callaghan.html

Open Science and peer-review in the humanities

Statut

“The purpose of this paper is to consider alternatives to the traditional system of peer review. I will argue that new methods of review should be more in accordance with the principles of Open Science. Current modes of carrying out peer review are functioning as barriers against more transparent ways of doing research. I will focus on peer reviewing as it is done in the humanities. These sciences seem to be clinging particularly tight to traditional ways of publishing and doing peer review. After looking at traditional peer review and the troubles related to it, I will discuss alternative ways of reviewing scholarly material. The anonymity of reviewers and authors, the appropriate time to make papers public, and how to reward reviewers are topics that are of importance in this context.”

URL : http://eprints.rclis.org/24136/