Étiquette : open science

Does the use of open, non-anonymous peer review in scholarly publishing introduce bias? Evidence from the F1000 post-publication open peer review publishing model

Authors : Mike Thelwall, Verena Weigert, Liz Allen, Zena Nyakoojo, Eleanor-Rose Papas

This study examines whether there is any evidence of bias in two areas of common critique of open, non-anonymous peer review – and used in the post-publication, peer review system operated by the open-access scholarly publishing platform F1000Research.

First, is there evidence of bias where a reviewer based in a specific country assesses the work of an author also based in the same country? Second, are reviewers influenced by being able to see the comments and know the origins of previous reviewer?

Scrutinising the open peer review comments published on F1000Research, we assess the extent of two frequently cited potential influences on reviewers that may be the result of the transparency offered by a fully attributable, open peer review publishing model: the national affiliations of authors and reviewers, and the ability of reviewers to view previously-published reviewer reports before submitting their own.

The effects of these potential influences were investigated for all first versions of articles published by 8 July 2019 to F1000Research. In 16 out of the 20 countries with the most articles, there was a tendency for reviewers based in the same country to give a more positive review.

The difference was statistically significant in one. Only 3 countries had the reverse tendency. Second, there is no evidence of a conformity bias. When reviewers mentioned a previous review in their peer review report, they were not more likely to give the same overall judgement.

Although reviewers who had longer to potentially read a previously published reviewer reports were slightly less likely to agree with previous reviewer judgements, this could be due to these articles being difficult to judge rather than deliberate non-conformity.

URL : https://arxiv.org/abs/1911.03379

Open Systems Pharmacology community – an open access, open source, open science approach to modeling and simulation in pharmaceutical sciences

Authors : Jörg Lippert, Rolf Burghaus, Andrea Edginton, Sebastian Frechen, Mats Karlsson, Andreas Kovar, Thorsten Lehr, Peter Milligan, Valerie Nock, Sergej Ramusovic, Matthew Riggs, Stephan Schaller, Jan Schlender, Stephan Schmidt, Michaël Sevestre, Erik Sjögren, Juri Solodenko, Alexander Staab, Donato Teutonico

Systems Pharmacology integrates structural biological and pharmacological knowledge and experimental data enabling dissection of organism and drug properties and providing excellent predictivity.

The development of systems pharmacology models is a significant task requiring massive amounts of background information beyond individual trial data. Qualification of models needs repetitive demonstration of successful predictions.

Open Systems Pharmacology is a community that develops, qualifies and shares professional open source software tools and models in a collaborative open science way.

DOI : https://doi.org/10.1002/psp4.12473

A Case Report: Building communities with training and resources for Open Science trainers

Authors: Helene Brinken, Iryna Kuchma, Vasso Kalaitzi, Joy Davidson, Nancy Pontika, Matteo Cancellieri, Antónia Correia, José Carvalho, Remedios Melero, Damjana Kastelic, Filomena Borba, Katerina Lenaki, Ulf Toelch, Katerina Zourou, Petr Knoth, Eloy Rodrigues

To foster responsible research and innovation, research communities, institutions, and funders are shifting their practices and requirements towards Open Science. Open Science skills are becoming increasingly essential for researchers. Indeed general awareness of Open Science has grown among EU researchers, but the practical adoption can be further improved.

Recognizing a gap between the needed and the provided training offer, the FOSTER project offers practical guidance and training to help researchers learn how to open up their research within a particular domain or research environment.

Aiming for a sustainable approach, FOSTER focused on strengthening the Open Science training capacity by establishing and supporting a community of trainers. The creation of an Open Science training handbook was a first step towards bringing together trainers to share their experiences and to create an open and living knowledge resource.

A subsequent series of train-the-trainer bootcamps helped trainers to find inspiration, improve their skills and to intensify exchange within a peer group. Four trainers, who attended one of the bootcamps, contributed a case study on their experiences and how they rolled out Open Science training within their own institutions.

On its platform the project provides a range of online courses and resources to learn about key Open Science topics. FOSTER awards users gamification badges when completing courses in order to provide incentives and rewards, and to spur them on to even greater achievements in learning.

The paper at hand describes FOSTER Plus’ training strategies, shares the lessons learnt and provides guidance on how to reuse the project’s materials and training approaches.

URL : A Case Report: Building communities with training and resources for Open Science trainers

DOI : http://doi.org/10.18352/lq.10303

Pubfair: a framework for sustainable, distributed, open science publishing services

Authors : Tony Ross-Hellauer, Benedikt Fecher, Kathleen Shearer, Eloy Rodrigues

Over the last thirty years, digitally-networked technologies have disrupted traditional media, turning business models on their head and changing the conditions for the creation, packaging and distribution of content.

Yet, scholarly communication still looks remarkably as it did in the pre-digital age. The primary unit of dissemination remains the research article (or book in some disciplines), and today’s articles still bear a remarkable resemblance to those that populated the pages of Oldenburg’s Philosophical Transactions 350 years ago.

In an age of such disruptive innovation, it is striking how little digital technologies have impacted scholarly publishing; and this is also somewhat ironic, since the Web was developed by scientists for research purposes.

Pubfair is a conceptual model for a modular open source publishing framework which builds upon a distributed network of repositories to enable the dissemination and quality-control of a range of research outputs including publications, data, and more.

Pubfair aims to introduce significant innovation into scholarly publishing. It enables different stakeholders (funders, institutions, scholarly societies, individuals scientists) to access a suite of functionalities to create their own dissemination channels, with built in open review and transparent processes.

The model minimizes publishing costs while maintaining academic standards by connecting communities with iterative publishing services linked to their preferred repository.

Such a publishing environment has the capacity to transform the scholarly communication system, making it more research-centric, dissemination-oriented and open to and supportive of innovation, while also collectively managed by the scholarly community.

URL : https://apo.org.au/node/257281

Bye Bye Peer-Reviewed Publishing

Authors : Miguel Abambres, Tony Salloom, Nejra Beganovic, Rafal Dojka, SergioRoncallo-Dow

This work is the continuation of a ‘revolution’ started with “Research Counts, Not the Journal”. Own and published opinions from worldwide scientists on critical issues of peer-reviewed publishing are presented.

In my opinion, peer-reviewed publishing is a quite flawed process (in many way) that has greatly harmed Science for a long time – it has been imposed by most academic and science funding institutions as the only way to assess scientific performance.

Unfortunately, most academics still follow that path, even though I believe most do it for the fear of losing their job or not being promoted. This paper aims to encourage (i) a full disruption of peer-reviewed publishing and (ii) the use of free eprint repositories for a sustainable academic/scientific publishing, i.e. healthier (no stress/distress associated to the peer review stage and the long waiting for publication) and more economic, effective and efficient (research is made immediately available and trackable/citable to anyone).

On the other hand, it should be pointed out that nothing exists against scientific publishers/journals – actually it´s perfectly normal that any company wants to implement its own quality criteria.

This paper is just the way chosen to promote the quick implementation of suitable policies for research evaluation.

URL : Bye Bye Peer-Reviewed Publishing

Alternative location : https://hal.archives-ouvertes.fr/hal-02114531v4

Open Up – the Mission Statement of the Control of Impulsive Action (Ctrl-ImpAct) Lab on Open Science

Authors : Christina B. Reimer, Zhang Chen, Carsten Bundt, Charlotte Eben, Raquel E. London, Sirarpi Vardanian

The present paper is the mission statement of the Control of Impulsive Action (Ctrl-ImpAct) Lab regarding Open Science. As early-career researchers (ECRs) in the lab, we first state our personal motivation to conduct research based on the principles of Open Science.

We then describe how we incorporate four specific Open Science practices (i.e., Open Methodology, Open Data, Open Source, and Open Access) into our scientific workflow. In more detail, we explain how Open Science practices are embedded into the so-called ‘co-pilot’ system in our lab.

The ‘co-pilot’ researcher is involved in all tasks of the ‘pilot’ researcher, that is designing a study, double-checking experimental and data analysis scripts, as well as writing the manuscript.

The lab has set up this co-pilot system to increase transparency, reduce potential errors that could occur during the entire workflow, and to intensify collaborations between lab members.

Finally, we discuss potential solutions for general problems that could arise when practicing Open Science.

URL : Open Up – the Mission Statement of the Control of Impulsive Action (Ctrl-ImpAct) Lab on Open Science

DOI : http://doi.org/10.5334/pb.494

Open Up: A Survey on Open and Non-anonymized Peer Reviewing

Authors : Lonni Besançon, Niklas Rönnberg, Jonas Löwgren, Jonathan P. Tennant, Matthew Cooper

We present a discussion and analysis regarding the benefits and limitations of open and non-anonymized peer review based on literature results and responses to a survey on the reviewing process of alt.chi, a more or less open-review track within the CHI conference, the predominant conference in the field of human-computer interaction (HCI).

This track currently is the only implementation of an open-peer-review process in the field of HCI while, with the recent increase in interest in open science practices, open review is now being considered and used in other fields.

We collected 30 responses from alt.chi authors and reviewers and found that, while the benefits are quite clear and the system is generally well liked by alt.chi participants, they are reluctant to see it used in other venues.

This concurs with a number of recent studies that suggest a divergence between support for a more open review process and its practical implementation. The data and scripts are available on https://osf.io/vuw7h/, and the figures and follow-up work on http://tiny.cc/OpenReviews.

URL : Open Up: A Survey on Open and Non-anonymized Peer Reviewing

Alternative location : https://www.preprints.org/manuscript/201905.0098/v2