Étiquette : Scholarly Publishing

Publié le

Over-optimization of academic publishing metrics: observing Goodhart’s Law in action

Authors : Michael Fire, Carlos Guestrin

Background

The academic publishing world is changing significantly, with ever-growing numbers of publications each year and shifting publishing patterns. However, the metrics used to measure academic success, such as the number of publications, citation number, and impact factor, have not changed for decades.

Moreover, recent studies indicate that these metrics have become targets and follow Goodhart’s Law, according to which, “when a measure becomes a target, it ceases to be a good measure.”

Results

In this study, we analyzed >120 million papers to examine how the academic publishing world has evolved over the last century, with a deeper look into the specific field of biology. Our study shows that the validity of citation-based measures is being compromised and their usefulness is lessening.

In particular, the number of publications has ceased to be a good metric as a result of longer author lists, shorter papers, and surging publication numbers. Citation-based metrics, such citation number and h-index, are likewise affected by the flood of papers, self-citations, and lengthy reference lists.

Measures such as a journal’s impact factor have also ceased to be good metrics due to the soaring numbers of papers that are published in top journals, particularly from the same pool of authors.

Moreover, by analyzing properties of >2,600 research fields, we observed that citation-based metrics are not beneficial for comparing researchers in different fields, or even in the same department.

Conclusions

Academic publishing has changed considerably; now we need to reconsider how we measure success.

URL : Over-optimization of academic publishing metrics: observing Goodhart’s Law in action

DOI : https://doi.org/10.1093/gigascience/giz053

Publié le

How to avoid borrowed plumes in academia

Authors : Margit Osterloh, Bruno S. Frey

Publications in top journals today have a powerful influence on academic careers although there is much criticism of using journal rankings to evaluate individual articles.

We ask why this practice of performance evaluation is still so influential. We suggest this is the case because a majority of authors benefit from the present system due to the extreme skewness of citation distributions. “Performance paradox” effects aggravate the problem.

Three extant suggestions for reforming performance management are critically discussed. We advance a new proposal based on the insight that fundamental uncertainty is symptomatic for scholarly work. It suggests focal randomization using a rationally founded and well-orchestrated procedure.

URL : How to avoid borrowed plumes in academia

Publié le

Revisiting ‘the 1990s debutante’: scholar-led publishing and the pre-history of the open access movement

Author : Samuel Moore

The movement for open access publishing is often said to have its roots in the scientific disciplines, having been popularised by scientific publishers and formalised through a range of top-down policy interventions. But there is an often-neglected pre-history of open access that can be found in the early DIY publishers of the late ‘80s and early ‘90s.

Managed entirely by working academics, these journals published research in the humanities and social sciences and stand out for their unique set of motivations and practices.

This article explores this separate lineage in the history of the open access movement through a critical-theoretical analysis of the motivations and practices of the early scholar-led publishers.

Alongside showing the involvement of the humanities and social sciences in the formation of open access, the analysis reveals the importance that these journals placed on experimental practices, critique of commercial publishing and the desire to reach new audiences.

Understood in today’s context, this research is significant for adding complexity to the history of open access, which policymakers, advocates and publishing scholars should keep in mind as open access goes mainstream.

DOI : http://dx.doi.org/10.17613/gty2-w177

Publié le

Ten Hot Topics around Scholarly Publishing

Authors : Jonathan P. Tennant, Harry Crane, Tom Crick, Jacinto Davila, Asura Enkhbayar, Johanna Havemann, Bianca Kramer, Ryan Martin, Paola Masuzzo,  Andy Nobes, Curt Rice, Bárbara Rivera-López, Tony Ross-Hellauer, Susanne Sattler, Paul D. Thacker, Marc Vanholsbeeck

The changing world of scholarly communication and the emerging new wave of ‘Open Science’ or ‘Open Research’ has brought to light a number of controversial and hotly debated topics.

Evidence-based rational debate is regularly drowned out by misinformed or exaggerated rhetoric, which does not benefit the evolving system of scholarly communication.

This article aims to provide a baseline evidence framework for ten of the most contested topics, in order to help frame and move forward discussions, practices, and policies.

We address issues around preprints and scooping, the practice of copyright transfer, the function of peer review, predatory publishers, and the legitimacy of ‘global’ databases.

These arguments and data will be a powerful tool against misinformation across wider academic research, policy and practice, and will inform changes within the rapidly evolving scholarly publishing system.

URL : Ten Hot Topics around Scholarly Publishing

DOI : https://doi.org/10.3390/publications7020034

Publié le

Article Processing Charge Hyperinflation and Price Insensitivity: An Open Access Sequel to the Serials Crisis

Author: Shaun Yon-Seng Khoo

Open access publishing has frequently been proposed as a solution to the serials crisis, which involved unsustainable budgetary pressures on libraries due to hyperinflation of subscription costs. The majority of open access articles are published in a minority of journals that levy article processing charges (APCs) paid by authors or their institutions upon acceptance.

Increases in APCs is proceeding at a rate three times that which would be expected if APCs were indexed according to inflation. As increasingly ambitious funder mandates are proposed, such as Plan S, it is important to evaluate whether authors show signs of price sensitivity in journal selection by avoiding journals that introduce or increase their APCs.

Examining journals that introduced an APC 4-5 years after launch or when flipping from a subscription model to immediate open access model showed no evidence that APC introduction reduced article volumes.

Multilevel modelling of APC sensitivity across 319 journals published by the four largest APC-funded dedicated commercial open access publishers (BMC, Frontiers, MDPI, and Hindawi) revealed that from 2012 to 2018 higher APCs were actually associated with increased article volumes.

These findings indicate that APC hyperinflation is not suppressed through market competition and author choice. Instead, demand for scholarly journal publications may be more similar to demand for necessities, or even prestige goods, which will support APC hyperinflation to the detriment of researchers, institutions, and funders.

URL : Article Processing Charge Hyperinflation and Price Insensitivity: An Open Access Sequel to the Serials Crisis

DOI : http://doi.org/10.18352/lq.10280

Publié le

Common Struggles: Policy-based vs. scholar-led approaches to open access in the humanities

Author : Samuel A. Moore

Open access publishing (OA) not only removes price and permission restrictions to academic research, but also represents an opportunity to reassess what publishing means to the humanities.

OA is increasingly on the agenda for humanities researchers in the UK, having been mandated in various forms by universities and governmental funders strongly influenced by advocates in the STEM disciplines.

Yet publishing practices in the humanities are unique to the field and any move to a new system of scholarly communication has the potential to conflict with the ways in which humanities research is published, many of which are shaped by the expectations of the neoliberal university that uniquely impact on the practices of humanities researchers.

Furthermore, OA does not reflect a unified ideology, business model or political outlook, and different methods of publication based on open practices will inherently represent a variety of values, struggles or conceptual enclosures.

This thesis assesses the contrasting values and practices of different approaches to OA in the humanities through a series of case-studies on governmental and scholar-led forms of OA, explored through a critical methodology comprising both constructivism and deconstruction.

The thesis argues that the UK governmental policy framework, comprised of policies introduced by the Research Councils (RCUK) and Higher Education Funding Councils (HEFCE), promotes a form of OA that intends to minimise disruption to the publishing industry.

The scholar-led ecosystem of presses, in contrast, reflects a diversity of values and struggles that represent a counter-hegemonic alternative to the dominant cultures of OA and publishing more generally.

The values of each approach are analysed on a spectrum between the logic of choice versus the logic of care (following the work of Annemarie Mol) to illustrate how the governmental policies promote a culture of OA predominantly focused on tangible outcomes, whereas the scholar-led presses prioritise an ethic of care for the cultures of how humanities research is produced and published.

In prioritising a commitment to care, scholar-led presses display a praxis that resembles the kinds of activities and relationships centred on common resource management (‘commoning’).

The thesis concludes with a series of recommendations for how such care-full values could be best realised in an emancipatory commons-based ecosystem of OA publishing for the humanities, which would be cultivated through a range of institutions and political interventions.

DOI : http://dx.doi.org/10.17613/st5m-cx33

Publié le

The Price of Gold: Curiosity?

Authors : Daniel W. Hook, Mark Hahnel, Christian Herzog

Gold open access as characterised by the payment of an article processing charge (APC) has become one of the dominant models in open access publication. This paper examines an extreme hypothetical case in which the APC model is the only model and the systematic issues that could develop in such a scenario.

URL : https://arxiv.org/abs/1905.00880