Bye Bye Peer-Reviewed Publishing

Authors : Miguel Abambres, Tony Salloom, Nejra Beganovic, Rafal Dojka, SergioRoncallo-Dow

This work is the continuation of a ‘revolution’ started with “Research Counts, Not the Journal”. Own and published opinions from worldwide scientists on critical issues of peer-reviewed publishing are presented.

In my opinion, peer-reviewed publishing is a quite flawed process (in many way) that has greatly harmed Science for a long time – it has been imposed by most academic and science funding institutions as the only way to assess scientific performance.

Unfortunately, most academics still follow that path, even though I believe most do it for the fear of losing their job or not being promoted. This paper aims to encourage (i) a full disruption of peer-reviewed publishing and (ii) the use of free eprint repositories for a sustainable academic/scientific publishing, i.e. healthier (no stress/distress associated to the peer review stage and the long waiting for publication) and more economic, effective and efficient (research is made immediately available and trackable/citable to anyone).

On the other hand, it should be pointed out that nothing exists against scientific publishers/journals – actually it´s perfectly normal that any company wants to implement its own quality criteria.

This paper is just the way chosen to promote the quick implementation of suitable policies for research evaluation.

URL : Bye Bye Peer-Reviewed Publishing

Alternative location : https://hal.archives-ouvertes.fr/hal-02114531v4

Open Up – the Mission Statement of the Control of Impulsive Action (Ctrl-ImpAct) Lab on Open Science

Authors : Christina B. Reimer, Zhang Chen, Carsten Bundt, Charlotte Eben, Raquel E. London, Sirarpi Vardanian

The present paper is the mission statement of the Control of Impulsive Action (Ctrl-ImpAct) Lab regarding Open Science. As early-career researchers (ECRs) in the lab, we first state our personal motivation to conduct research based on the principles of Open Science.

We then describe how we incorporate four specific Open Science practices (i.e., Open Methodology, Open Data, Open Source, and Open Access) into our scientific workflow. In more detail, we explain how Open Science practices are embedded into the so-called ‘co-pilot’ system in our lab.

The ‘co-pilot’ researcher is involved in all tasks of the ‘pilot’ researcher, that is designing a study, double-checking experimental and data analysis scripts, as well as writing the manuscript.

The lab has set up this co-pilot system to increase transparency, reduce potential errors that could occur during the entire workflow, and to intensify collaborations between lab members.

Finally, we discuss potential solutions for general problems that could arise when practicing Open Science.

URL : Open Up – the Mission Statement of the Control of Impulsive Action (Ctrl-ImpAct) Lab on Open Science

DOI : http://doi.org/10.5334/pb.494

Open Up: A Survey on Open and Non-anonymized Peer Reviewing

Authors : Lonni Besançon, Niklas Rönnberg, Jonas Löwgren, Jonathan P. Tennant, Matthew Cooper

We present a discussion and analysis regarding the benefits and limitations of open and non-anonymized peer review based on literature results and responses to a survey on the reviewing process of alt.chi, a more or less open-review track within the CHI conference, the predominant conference in the field of human-computer interaction (HCI).

This track currently is the only implementation of an open-peer-review process in the field of HCI while, with the recent increase in interest in open science practices, open review is now being considered and used in other fields.

We collected 30 responses from alt.chi authors and reviewers and found that, while the benefits are quite clear and the system is generally well liked by alt.chi participants, they are reluctant to see it used in other venues.

This concurs with a number of recent studies that suggest a divergence between support for a more open review process and its practical implementation. The data and scripts are available on https://osf.io/vuw7h/, and the figures and follow-up work on http://tiny.cc/OpenReviews.

URL : Open Up: A Survey on Open and Non-anonymized Peer Reviewing

Alternative location : https://www.preprints.org/manuscript/201905.0098/v2

 

Open science and modified funding lotteries can impede the natural selection of bad science

Authors : Paul E. Smaldino, Matthew A. Turner, Pablo A. Contreras Kallens

Assessing scientists using exploitable metrics can lead to the degradation of research methods even without any strategic behaviour on the part of individuals, via ‘the natural selection of bad science.’

Institutional incentives to maximize metrics like publication quantity and impact drive this dynamic. Removing these incentives is necessary, but institutional change is slow.

However, recent developments suggest possible solutions with more rapid onsets. These include what we call open science improvements, which can reduce publication bias and improve the efficacy of peer review. In addition, there have been increasing calls for funders to move away from prestige- or innovation-based approaches in favour of lotteries.

We investigated whether such changes are likely to improve the reproducibility of science even in the presence of persistent incentives for publication quantity through computational modelling.

We found that modified lotteries, which allocate funding randomly among proposals that pass a threshold for methodological rigour, effectively reduce the rate of false discoveries, particularly when paired with open science improvements that increase the publication of negative results and improve the quality of peer review.

In the absence of funding that targets rigour, open science improvements can still reduce false discoveries in the published literature but are less likely to improve the overall culture of research practices that underlie those publications.

URL : Open science and modified funding lotteries can impede the natural selection of bad science

DOI : https://doi.org/10.1098/rsos.190194

Open Social Scholarship Annotated Bibliography

Authors: Randa El Khatib, Lindsey Seatter, Tracey El Hajj, Conrad Leibel, Alyssa Arbuckle, Ray Siemens, Caroline Winter, the ETCL and INKE Research Groups

This annotated bibliography responds to and contextualizes the growing “Open” movements and recent institutional reorientation towards social, public-facing scholarship.

The aim of this document is to present a working definition of open social scholarship through the aggregation and summation of critical resources in the field. Our work surveys foundational publications, innovative research projects, and global organizations that enact the theories and practices of open social scholarship.

The bibliography builds on the knowledge creation principles outlined in previous research by broadening the focus beyond traditional academic spaces and reinvigorating central, defining themes with recently published research.

URL : Open Social Scholarship Annotated Bibliography

DOI : http://doi.org/10.5334/kula.58

The Economic Impacts of Open Science: A Rapid Evidence Assessment

Author : Michael J. Fell

A common motivation for increasing open access to research findings and data is the potential to create economic benefits—but evidence is patchy and diverse. This study systematically reviewed the evidence on what kinds of economic impacts (positive and negative) open science can have, how these comes about, and how benefits could be maximized.

Use of open science outputs often leaves no obvious trace, so most evidence of impacts is based on interviews, surveys, inference based on existing costs, and modelling approaches.

There is indicative evidence that open access to findings/data can lead to savings in access costs, labour costs and transaction costs. There are examples of open science enabling new products, services, companies, research and collaborations. Modelling studies suggest higher returns to R&D if open access permits greater accessibility and efficiency of use of findings. Barriers include lack of skills capacity in search, interpretation and text mining, and lack of clarity around where benefits accrue.

There are also contextual considerations around who benefits most from open science (e.g., sectors, small vs. larger companies, types of dataset). Recommendations captured in the review include more research, monitoring and evaluation (including developing metrics), promoting benefits, capacity building and making outputs more audience-friendly.

URL : The Economic Impacts of Open Science: A Rapid Evidence Assessment

DOI : https://doi.org/10.3390/publications7030046

The Definition of Reuse

Authors : Stephanie van de Sandt, Sünje Dallmeier-Tiessen, Artemis Lavasa, Vivien Petras

The ability to reuse research data is now considered a key benefit for the wider research community. Researchers of all disciplines are confronted with the pressure to share their research data so that it can be reused.

The demand for data use and reuse has implications on how we document, publish and share research in the first place, and, perhaps most importantly, it affects how we measure the impact of research, which is commonly a measurement of its use and reuse.

It is surprising that research communities, policy makers, etc. have not clearly defined what use and reuse is yet.

We postulate that a clear definition of use and reuse is needed to establish better metrics for a comprehensive scholarly record of individuals, institutions, organizations, etc.

Hence, this article presents a first definition of reuse of research data. Characteristics of reuse are identified by examining the etymology of the term and the analysis of the current discourse, leading to a range of reuse scenarios that show the complexity of today’s research landscape, which has been moving towards a data-driven approach.

The analysis underlines that there is no reason to distinguish use and reuse. We discuss what that means for possible new metrics that attempt to cover Open Science practices more comprehensively.

We hope that the resulting definition will enable a better and more refined strategy for Open Science.

URL : The Definition of Reuse

DOI : http://doi.org/10.5334/dsj-2019-022