Archives des mots-clés : Scholarly Publishing

The F3-index. Valuing reviewers for scholarly journals

Posté le par

Authors : Federico Bianchi, Francisco Grimaldo, Flaminio Squazzoni

This paper presents an index that measures reviewer contribution to editorial processes of scholarly journals. Following a metaphor of ranking algorithms in sports tournaments, we created an index that considers reviewers on different context-specific dimensions, i.e., report delivery time, the length of the report and the alignment of recommendations to editorial decisions.

To test the index, we used a dataset of peer review in a multi-disciplinary journal, including 544 reviewers on 606 submissions in six years. Although limited by sample size, the test showed that the index identifies outstanding contributors and weak performing reviewers efficiently.

Our index is flexible, contemplates extensions and could be incorporated into available scholarly journal management tools. It can assist editors in rewarding high performing reviewers and managing editorial turnover.

URL : The F3-index. Valuing reviewers for scholarly journals

DOI : https://doi.org/10.1016/j.joi.2018.11.007

“Blacklists” and “whitelists” to tackle predatory publishing : A cross-sectional comparison and thematic analysis

Posté le par

Authors : Michaela Strinzel​, Anna Severin, Katrin Milzow, Matthias Egger

Background

Despite growing awareness of predatory publishing and research on its market characteristics, the defining attributes of fraudulent journals remain controversial.

We aimed to develop a better understanding of quality criteria for scholarly journals by analysing journals and publishers indexed in blacklists of predatory journals and whitelists of legitimate journals and the lists’ inclusion criteria.

Methods

We searched for blacklists and whitelists in early 2018. Lists that included journals across disciplines were eligible. We used a mixed methods approach, combining quantitative and qualitative analyses.

To quantify overlaps between lists in terms of indexed journals and publishers we employed the Jaro-Winkler string metric and Venn diagrams. To identify topics addressed by the lists’ inclusion criteria and to derive their broader conceptual categories, we used a qualitative coding approach.

Results

Two blacklists (Beall’s and Cabell’s) and two whitelists (DOAJ and Cabell’s) were eligible. The number of journals per list ranged from 1404 to 12357 and the number of publishers from 473 to 5638. Seventy-three journals and 42 publishers were included both in a blacklist and whitelist. A total of 198 inclusion criteria were examined.

Seven thematic themes were identified: (i) peer review, (ii) editorial services, (iii) policy, (iv) business practices, (v) publishing, archiving and access, (vi) website and (vii) indexing and metrics.

Business practices accounted for almost half of blacklists’ criteria, whereas whitelists gave more emphasis to criteria related to policy and guidelines. Criteria were grouped into four broad concepts: (i) transparency, (ii) ethics, (iii) professional standards and (iv) peer review and other services.

Whitelists gave more weight to transparency whereas blacklists focused on ethics and professional standards. The criteria included in whitelists were easier to verify than those used in blacklists. Both types of list gave relatively little emphasis to the quality of peer review.

Conclusions

There is overlap between journals and publishers included in blacklists and whitelists. Blacklists and whitelists differ in their criteria for quality and the weight given to different dimensions of quality. Aspects that are central but difficult to verify receive insufficient attention.

URL : “Blacklists” and “whitelists” to tackle predatory publishing : A cross-sectional comparison and thematic analysis

DOI : https://doi.org/10.7287/peerj.preprints.27532v1

Ten years of AoB PLANTS the open access journal for plant scientists: inception and progress since 2009

Posté le par

Author : Michael B Jackson

AoB PLANTS is a not-for-profit, open access, plant science journal and one of three peer-reviewed journals owned and managed by the Annals of Botany Company. This article explains events and thinking that led to the starting of AoB PLANTS and how the unique features of the Journal came to be formalized prior to its launch in September 2009.

The article also describes how the Journal’s management developed over the first 10 years and summarizes the Journal’s achievements in a decade where open access journals have proliferated despite subscription journals continuing to dominate the publishing of peer-reviewed botanical science.

URL : Ten years of AoB PLANTS the open access journal for plant scientists: inception and progress since 2009

DOI : https://doi.org/10.1093/aobpla/plz025

Over-optimization of academic publishing metrics: observing Goodhart’s Law in action

Posté le par

Authors : Michael Fire, Carlos Guestrin

Background

The academic publishing world is changing significantly, with ever-growing numbers of publications each year and shifting publishing patterns. However, the metrics used to measure academic success, such as the number of publications, citation number, and impact factor, have not changed for decades.

Moreover, recent studies indicate that these metrics have become targets and follow Goodhart’s Law, according to which, “when a measure becomes a target, it ceases to be a good measure.”

Results

In this study, we analyzed >120 million papers to examine how the academic publishing world has evolved over the last century, with a deeper look into the specific field of biology. Our study shows that the validity of citation-based measures is being compromised and their usefulness is lessening.

In particular, the number of publications has ceased to be a good metric as a result of longer author lists, shorter papers, and surging publication numbers. Citation-based metrics, such citation number and h-index, are likewise affected by the flood of papers, self-citations, and lengthy reference lists.

Measures such as a journal’s impact factor have also ceased to be good metrics due to the soaring numbers of papers that are published in top journals, particularly from the same pool of authors.

Moreover, by analyzing properties of >2,600 research fields, we observed that citation-based metrics are not beneficial for comparing researchers in different fields, or even in the same department.

Conclusions

Academic publishing has changed considerably; now we need to reconsider how we measure success.

URL : Over-optimization of academic publishing metrics: observing Goodhart’s Law in action

DOI : https://doi.org/10.1093/gigascience/giz053

How to avoid borrowed plumes in academia

Posté le par

Authors : Margit Osterloh, Bruno S. Frey

Publications in top journals today have a powerful influence on academic careers although there is much criticism of using journal rankings to evaluate individual articles.

We ask why this practice of performance evaluation is still so influential. We suggest this is the case because a majority of authors benefit from the present system due to the extreme skewness of citation distributions. “Performance paradox” effects aggravate the problem.

Three extant suggestions for reforming performance management are critically discussed. We advance a new proposal based on the insight that fundamental uncertainty is symptomatic for scholarly work. It suggests focal randomization using a rationally founded and well-orchestrated procedure.

URL : How to avoid borrowed plumes in academia

Revisiting ‘the 1990s debutante’: scholar-led publishing and the pre-history of the open access movement

Posté le par

Author : Samuel Moore

The movement for open access publishing is often said to have its roots in the scientific disciplines, having been popularised by scientific publishers and formalised through a range of top-down policy interventions. But there is an often-neglected pre-history of open access that can be found in the early DIY publishers of the late ‘80s and early ‘90s.

Managed entirely by working academics, these journals published research in the humanities and social sciences and stand out for their unique set of motivations and practices.

This article explores this separate lineage in the history of the open access movement through a critical-theoretical analysis of the motivations and practices of the early scholar-led publishers.

Alongside showing the involvement of the humanities and social sciences in the formation of open access, the analysis reveals the importance that these journals placed on experimental practices, critique of commercial publishing and the desire to reach new audiences.

Understood in today’s context, this research is significant for adding complexity to the history of open access, which policymakers, advocates and publishing scholars should keep in mind as open access goes mainstream.

DOI : http://dx.doi.org/10.17613/gty2-w177

Ten Hot Topics around Scholarly Publishing

Posté le par

Authors : Jonathan P. Tennant, Harry Crane, Tom Crick, Jacinto Davila, Asura Enkhbayar, Johanna Havemann, Bianca Kramer, Ryan Martin, Paola Masuzzo,  Andy Nobes, Curt Rice, Bárbara Rivera-López, Tony Ross-Hellauer, Susanne Sattler, Paul D. Thacker, Marc Vanholsbeeck

The changing world of scholarly communication and the emerging new wave of ‘Open Science’ or ‘Open Research’ has brought to light a number of controversial and hotly debated topics.

Evidence-based rational debate is regularly drowned out by misinformed or exaggerated rhetoric, which does not benefit the evolving system of scholarly communication.

This article aims to provide a baseline evidence framework for ten of the most contested topics, in order to help frame and move forward discussions, practices, and policies.

We address issues around preprints and scooping, the practice of copyright transfer, the function of peer review, predatory publishers, and the legitimacy of ‘global’ databases.

These arguments and data will be a powerful tool against misinformation across wider academic research, policy and practice, and will inform changes within the rapidly evolving scholarly publishing system.

URL : Ten Hot Topics around Scholarly Publishing

DOI : https://doi.org/10.3390/publications7020034