Étiquette : Open peer review

Publié le

Automatically Annotating Articles Towards Opening and Reusing Transparent Peer Reviews

Authors : Afshin Sadeghi, Sarven Capadisli, Johannes Wilm, Christoph Lange, Philipp Mayr

An increasing number of scientific publications are created in open and transparent peer review models: a submission is published first, and then reviewers are invited, or a submission is reviewed in a closed environment but then these reviews are published with the final article, or combinations of these.

Reasons for open peer review include giving better credit to reviewers and enabling readers to better appraise the quality of a publication. In most cases, the full, unstructured text of an open review is published next to the full, unstructured text of the article reviewed.

This approach prevents human readers from getting a quick impression of the quality of parts of an article, and it does not easily support secondary exploitation, e.g., for scientometrics on reviews.

While document formats have been proposed for publishing structured articles including reviews, integrated tool support for entire open peer review workflows resulting in such documents is still scarce.

We present AR-Annotator, the Automatic Article and Review Annotator which employs a semantic information model of an article and its reviews, using semantic markup and unique identifiers for all entities of interest.

The fine-grained article structure is not only exposed to authors and reviewers but also preserved in the published version. We publish articles and their reviews in a Linked Data representation and thus maximize their reusability by third-party applications.

We demonstrate this reusability by running quality-related queries against the structured representation of articles and their reviews.

URL : https://arxiv.org/abs/1812.01027

Publié le

Reviewing the review process: Investigation of researchers’ opinions on different methods of peer review

Author : Carolin Anna Rebernig

Peer review is considered the gold standard of scientific publishing. Trust in the traditional system of editor – blind-reviewer – author is still high, but it’s authority is in decline and alternative methods are on the rise.

The current study investigates opinions of alternative peer review methods, the arguments for and against, and the reasons why academics are searching for new approaches. The opinions were analysed by applying qualitative content analysis to online discussions.

The findings were interpreted using two different sociological theories: the Mertonian sociology of science and social constructivism.

The results of the study show that the most discussed method was also the most traditional one: closed pre-publication peer review comprised of single blind, double-blind and open peer review (non blinded).

Discussions of open peer review (both open publishing of reports and open discussions) were also common. All other alternative methods were discussed much less. But the discussions were lively and each method was discussed in both positive and negative terms.

The reasons for preferring certain methods were also manifold, but dominant topics were bias and fairness, quality issues (regarding reviews and publications), issues concerning human resources and communication and exchange among people.

The results of this study demonstrate that while ethical norms seems to be a scientific ideal, human nature makes it impossible to accomplish this goal.

URN : http://urn.kb.se/resolve?urn=urn%3Anbn%3Ase%3Ahb%3Adiva-14607

Publié le

Scholarship as an Open Conversation: Utilizing Open Peer Review in Information Literacy Instruction

Author : Emily Ford

This article explores the ACRL Framework for Information Literacy’s frame, Scholarship as a Conversation. This frame asserts that information literate students have the disposition, skills, and knowledge to recognize and participate in disciplinary scholarly conversations.

By investigating the peer-review process as part of scholarly conversations, this article provides a brief literature review on peer review in information literacy instruction, and argues that by using open peer review (OPR) models for teaching, library workers can allow students to gain a deeper understanding of scholarly conversations.

OPR affords students the ability to begin dismantling the systemic oppression that blinded peer review and the traditional scholarly publishing system reinforce. Finally, the article offers an example classroom activity using OPR to help students enter scholarly conversations, and recognize power and oppression in scholarly publishing.

URL : http://www.inthelibrarywiththeleadpipe.org/2018/open-conversation/

Publié le

Ten considerations for open peer review

Authors : Birgit Schmidt, Tony Ross-Hellauer, Xenia van Edig, Elizabeth C Moylan

Open peer review (OPR), as with other elements of open science and open research, is on the rise. It aims to bring greater transparency and participation to formal and informal peer review processes.

But what is meant by `open peer review’, and what advantages and disadvantages does it have over standard forms of review? How do authors or reviewers approach OPR? And what pitfalls and opportunities should you look out for?

Here, we propose ten considerations for OPR, drawing on discussions with authors, reviewers, editors, publishers and librarians, and provide a pragmatic, hands-on introduction to these issues.

We cover basic principles and summarise best practices, indicating how to use OPR to achieve best value and mutual benefits for all stakeholders and the wider research community.

URL : Ten considerations for open peer review

DOI : http://dx.doi.org/10.12688/f1000research.15334.1

Publié le

Developing indicators on Open Access by combining evidence from diverse data sources

Authors : Thed van Leeuwen, Ingeborg Meijer, Alfredo Yegros-Yegros, Rodrigo Costas

In the last couple of years, the role of Open Access (OA) publishing has become central in science management and research policy. In the UK and the Netherlands, national OA mandates require the scientific community to seriously consider publishing research outputs in OA forms.

At the same time, other elements of Open Science are becoming also part of the debate, thus including not only publishing research outputs but also other related aspects of the chain of scientific knowledge production such as open peer review and open data.

From a research management point of view, it is important to keep track of the progress made in the OA publishing debate. Until now, this has been quite problematic, given the fact that OA as a topic is hard to grasp by bibliometric methods, as most databases supporting bibliometric data lack exhaustive and accurate open access labelling of scientific publications.

In this study, we present a methodology that systematically creates OA labels for large sets of publications processed in the Web of Science database. The methodology is based on the combination of diverse data sources that provide evidence of publications being OA.

URL : https://arxiv.org/abs/1802.02827v1

Publié le

Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers

Authors : Tony Ross-Hellauer, Arvid Deppe, Birgit Schmidt

Open peer review (OPR) is a cornerstone of the emergent Open Science agenda. Yet to date no large-scale survey of attitudes towards OPR amongst academic editors, authors, reviewers and publishers has been undertaken.

This paper presents the findings of an online survey, conducted for the OpenAIRE2020 project during September and October 2016, that sought to bridge this information gap in order to aid the development of appropriate OPR approaches by providing evidence about attitudes towards and levels of experience with OPR.

The results of this cross-disciplinary survey, which received 3,062 full responses, show the majority (60.3%) of respondents to be believe that OPR as a general concept should be mainstream scholarly practice (although attitudes to individual traits varied, and open identities peer review was not generally favoured). Respondents were also in favour of other areas of Open Science, like Open Access (88.2%) and Open Data (80.3%).

Among respondents we observed high levels of experience with OPR, with three out of four (76.2%) reporting having taken part in an OPR process as author, reviewer or editor.

There were also high levels of support for most of the traits of OPR, particularly open interaction, open reports and final-version commenting. Respondents were against opening reviewer identities to authors, however, with more than half believing it would make peer review worse.

Overall satisfaction with the peer review system used by scholarly journals seems to strongly vary across disciplines. Taken together, these findings are very encouraging for OPR’s prospects for moving mainstream but indicate that due care must be taken to avoid a “one-size fits all” solution and to tailor such systems to differing (especially disciplinary) contexts.

OPR is an evolving phenomenon and hence future studies are to be encouraged, especially to further explore differences between disciplines and monitor the evolution of attitudes.

URL : Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers

DOI : https://doi.org/10.1371/journal.pone.0189311

Publié le

A prospective study on an innovative online forum for peer reviewing of surgical science

Authors : Martin Almquist, Regula S. von Allmen, Dan Carradice, Steven J. Oosterling, Kirsty McFarlane, Bas Wijnhoven

Background

Peer review is important to the scientific process. However, the present system has been criticised and accused of bias, lack of transparency, failure to detect significant breakthrough and error. At the British Journal of Surgery (BJS), after surveying authors’ and reviewers’ opinions on peer review, we piloted an open online forum with the aim of improving the peer review process.

Methods

In December 2014, a web-based survey assessing attitudes towards open online review was sent to reviewers with a BJS account in Scholar One. From April to June 2015, authors were invited to allow their manuscripts to undergo online peer review in addition to the standard peer review process.

The quality of each review was evaluated by editors and editorial assistants using a validated instrument based on a Likert scale.

Results

The survey was sent to 6635 reviewers. In all, 1454 (21.9%) responded. Support for online peer review was strong, with only 10% stating that they would not subject their manuscripts to online peer review. The most prevalent concern was about intellectual property, being highlighted in 118 of 284 comments (41.5%).

Out of 265 eligible manuscripts, 110 were included in the online peer review trial. Around 7000 potential reviewers were invited to review each manuscript.

In all, 44 of 110 manuscripts (40%) received 100 reviews from 59 reviewers, alongside 115 conventional reviews. The quality of the open forum reviews was lower than for conventional reviews (2.13 (± 0.75) versus 2.84 (± 0.71), P<0.001).

Conclusion

Open online peer review is feasible in this setting, but it attracts few reviews, of lower quality than conventional peer reviews.

URL : A prospective study on an innovative online forum for peer reviewing of surgical science

DOI : https://doi.org/10.1371/journal.pone.0179031