Developing indicators on Open Access by combining evidence from diverse data sources

Authors : Thed van Leeuwen, Ingeborg Meijer, Alfredo Yegros-Yegros, Rodrigo Costas

In the last couple of years, the role of Open Access (OA) publishing has become central in science management and research policy. In the UK and the Netherlands, national OA mandates require the scientific community to seriously consider publishing research outputs in OA forms.

At the same time, other elements of Open Science are becoming also part of the debate, thus including not only publishing research outputs but also other related aspects of the chain of scientific knowledge production such as open peer review and open data.

From a research management point of view, it is important to keep track of the progress made in the OA publishing debate. Until now, this has been quite problematic, given the fact that OA as a topic is hard to grasp by bibliometric methods, as most databases supporting bibliometric data lack exhaustive and accurate open access labelling of scientific publications.

In this study, we present a methodology that systematically creates OA labels for large sets of publications processed in the Web of Science database. The methodology is based on the combination of diverse data sources that provide evidence of publications being OA.


Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers

Authors : Tony Ross-Hellauer, Arvid Deppe, Birgit Schmidt

Open peer review (OPR) is a cornerstone of the emergent Open Science agenda. Yet to date no large-scale survey of attitudes towards OPR amongst academic editors, authors, reviewers and publishers has been undertaken.

This paper presents the findings of an online survey, conducted for the OpenAIRE2020 project during September and October 2016, that sought to bridge this information gap in order to aid the development of appropriate OPR approaches by providing evidence about attitudes towards and levels of experience with OPR.

The results of this cross-disciplinary survey, which received 3,062 full responses, show the majority (60.3%) of respondents to be believe that OPR as a general concept should be mainstream scholarly practice (although attitudes to individual traits varied, and open identities peer review was not generally favoured). Respondents were also in favour of other areas of Open Science, like Open Access (88.2%) and Open Data (80.3%).

Among respondents we observed high levels of experience with OPR, with three out of four (76.2%) reporting having taken part in an OPR process as author, reviewer or editor.

There were also high levels of support for most of the traits of OPR, particularly open interaction, open reports and final-version commenting. Respondents were against opening reviewer identities to authors, however, with more than half believing it would make peer review worse.

Overall satisfaction with the peer review system used by scholarly journals seems to strongly vary across disciplines. Taken together, these findings are very encouraging for OPR’s prospects for moving mainstream but indicate that due care must be taken to avoid a “one-size fits all” solution and to tailor such systems to differing (especially disciplinary) contexts.

OPR is an evolving phenomenon and hence future studies are to be encouraged, especially to further explore differences between disciplines and monitor the evolution of attitudes.

URL : Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers


A prospective study on an innovative online forum for peer reviewing of surgical science

Authors : Martin Almquist, Regula S. von Allmen, Dan Carradice, Steven J. Oosterling, Kirsty McFarlane, Bas Wijnhoven


Peer review is important to the scientific process. However, the present system has been criticised and accused of bias, lack of transparency, failure to detect significant breakthrough and error. At the British Journal of Surgery (BJS), after surveying authors’ and reviewers’ opinions on peer review, we piloted an open online forum with the aim of improving the peer review process.


In December 2014, a web-based survey assessing attitudes towards open online review was sent to reviewers with a BJS account in Scholar One. From April to June 2015, authors were invited to allow their manuscripts to undergo online peer review in addition to the standard peer review process.

The quality of each review was evaluated by editors and editorial assistants using a validated instrument based on a Likert scale.


The survey was sent to 6635 reviewers. In all, 1454 (21.9%) responded. Support for online peer review was strong, with only 10% stating that they would not subject their manuscripts to online peer review. The most prevalent concern was about intellectual property, being highlighted in 118 of 284 comments (41.5%).

Out of 265 eligible manuscripts, 110 were included in the online peer review trial. Around 7000 potential reviewers were invited to review each manuscript.

In all, 44 of 110 manuscripts (40%) received 100 reviews from 59 reviewers, alongside 115 conventional reviews. The quality of the open forum reviews was lower than for conventional reviews (2.13 (± 0.75) versus 2.84 (± 0.71), P<0.001).


Open online peer review is feasible in this setting, but it attracts few reviews, of lower quality than conventional peer reviews.

URL : A prospective study on an innovative online forum for peer reviewing of surgical science



What is open peer review? A systematic review

Author : Tony Ross-Hellauer


“Open peer review” (OPR), despite being a major pillar of Open Science, has neither a standardized definition nor an agreed schema of its features and implementations. The literature reflects this, with a myriad of overlapping and often contradictory definitions.

While the term is used by some to refer to peer review where the identities of both author and reviewer are disclosed to each other, for others it signifies systems where reviewer reports are published alongside articles.

For others it signifies both of these conditions, and for yet others it describes systems where not only “invited experts” are able to comment. For still others, it includes a variety of combinations of these and other novel methods.


Recognising the absence of a consensus view on what open peer review is, this article undertakes a systematic review of definitions of “open peer review” or “open review”, to create a corpus of 122 definitions.

These definitions are then systematically analysed to build a coherent typology of the many different innovations in peer review signified by the term, and hence provide the precise technical definition currently lacking.


This quantifiable data yields rich information on the range and extent of differing definitions over time and by broad subject area. Quantifying definitions in this way allows us to accurately portray exactly how  ambiguously the phrase “open peer review”  has been used thus far, for the literature offers a total of 22 distinct configurations of seven traits, effectively meaning that there are 22 different definitions of OPR in the literature.


Based on this work, I propose a pragmatic definition of open peer review as an umbrella term for a number of overlapping ways that peer review models can be adapted in line with the ethos of Open Science, including making reviewer and author identities open, publishing review reports and enabling greater participation in the peer review process.

URL : What is open peer review? A systematic review


Emerging trends in peer review—a survey

Authors : Richard Walker, Pascal Rocha da Silva

“Classical peer review” has been subject to intense criticism for slowing down the publication process, bias against specific categories of paper and author, unreliability, inability to detect errors and fraud, unethical practices, and the lack of recognition for unpaid reviewers.

This paper surveys innovative forms of peer review that attempt to address these issues. Based on an initial literature review, we construct a sample of 82 channels of scientific communication covering all forms of review identified by the survey, and analyze the review mechanisms used by each channel.

We identify two major trends: the rapidly expanding role of preprint servers (e.g., ArXiv) that dispense with traditional peer review altogether, and the growth of “non-selective review,” focusing on papers’ scientific quality rather than their perceived importance and novelty.

Other potentially important developments include forms of “open review,” which remove reviewer anonymity, and interactive review, as well as new mechanisms for post-publication review and out-of-channel reader commentary, especially critical commentary targeting high profile papers.

One of the strongest findings of the survey is the persistence of major differences between the peer review processes used by different disciplines. None of these differences is likely to disappear in the foreseeable future.

The most likely scenario for the coming years is thus continued diversification, in which different review mechanisms serve different author, reader, and publisher needs. Relatively little is known about the impact of these innovations on the problems they address. These are important questions for future quantitative research.

URL : Emerging trends in peer review—a survey


Open Post Publication Peer Review: An Innovation in Scientific Communication

Authors : Peiling Wang, Manasa Rath, Michael Deike, Wu Qiang

This research observes the emerging open peer review journals. In scientific publishing, transparency in peer review is a growing topic of interest for online journals. The traditional blind refereeing process has been criticized for lacking transparency.

Although the idea of open peer review (OPR) has been explored since 1980s, it is only in this decade that OPR journals are born. Towards a more open publishing model, the peer review process–once accessible only to the editors and referees—is now available to public.

The published article and its review history are being integrated into one entity; readers can submit or post comments to extend the peer process. This preliminary study observed four pioneer OPR journals representing pre-publication OPR and post-publication OPR.

Data collection focuses on publication’s lifecycle from its submission to peer approval. Preliminary results include comparisons of the level of openness and nature of interactions during refereeing process.


Opening Review in LIS Journals: A Status Report

Author : Emily Ford


Peer-review practices in scholarly publishing are changing. Digital publishing mechanisms allow for open peer review, a peer review process that discloses author and reviewer identities to one another.

This model of peer review is increasingly implemented in scholarly publishing. In science, technology, engineering, and math (STEM) disciplines, open peer review is implemented in journal publishing processes, and, in the humanities and social sciences, it is often coupled with new scholarship practices, such as the digital humanities.

This article reports findings from an exploratory study on peer-review and publishing practices in Library and Information Science (LIS), focusing on LIS’s relationships with open peer review.


Editors of LIS journals were surveyed regarding journal peer review and publishing practices.


This article reports the general “pulse” of attitudes and conversations regarding open peer review and discusses its challenges in LIS. Results show an ideological split between traditionally-published journals and open access and association-affiliated journals. Open access and association-affiliated journal editors are more likely to consider investigating open peer review.


The LIS community of journal editors, authors, reviewers, and readers need to discuss open peer review as well as experiment with it. Experiments with open peer review in scholarly LIS publishing will inform our praxis as librarians.

URL : Opening Review in LIS Journals: A Status Report