Étiquette : Lisa Matthias

Meta-Research: Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations

Authors : Erin C McKiernan, Lesley A Schimanski, Carol Muñoz Nieves, Lisa Matthias, Meredith T Niles, Juan P Alperin

We analyzed how often and in what ways the Journal Impact Factor (JIF) is currently used in review, promotion, and tenure (RPT) documents of a representative sample of universities from the United States and Canada. 40% of research-intensive institutions and 18% of master’s institutions mentioned the JIF, or closely related terms.

Of the institutions that mentioned the JIF, 87% supported its use in at least one of their RPT documents, 13% expressed caution about its use, and none heavily criticized it or prohibited its use. Furthermore, 63% of institutions that mentioned the JIF associated the metric with quality, 40% with impact, importance, or significance, and 20% with prestige, reputation, or status.

We conclude that use of the JIF is encouraged in RPT evaluations, especially at research-intensive universities, and that there is work to be done to avoid the potential misuse of metrics like the JIF.

URL : Meta-Research: Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations

DOI : https://doi.org/10.7554/eLife.47338.001

Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations

Authors : Erin C. McKiernan​, Lesley A. Schimanski, Carol Muñoz Nieves, Lisa Matthias, Meredith T. Niles, Juan Pablo Alperin

The Journal Impact Factor (JIF) was originally designed to aid libraries in deciding which journals to index and purchase for their collections. Over the past few decades, however, it has become a relied upon metric used to evaluate research articles based on journal rank. Surveyed faculty often report feeling pressure to publish in journals with high JIFs and mention reliance on the JIF as one problem with current academic evaluation systems.

While faculty reports are useful, information is lacking on how often and in what ways the JIF is currently used for review, promotion, and tenure (RPT). We therefore collected and analyzed RPT documents from a representative sample of 129 universities from the United States and Canada and 381 of their academic units.

We found that 40% of doctoral, research-intensive (R-type) institutions and 18% of master’s, or comprehensive (M-type) institutions explicitly mentioned the JIF, or closely related terms, in their RPT documents.

Undergraduate, or baccalaureate (B-type) institutions did not mention it at all. A detailed reading of these documents suggests that institutions may also be using a variety of terms to indirectly refer to the JIF.

Our qualitative analysis shows that 87% of the institutions that mentioned the JIF supported the metric’s use in at least one of their RPT documents, while 13% of institutions expressed caution about the JIF’s use in evaluations.

None of the RPT documents we analyzed heavily criticized the JIF or prohibited its use in evaluations. Of the institutions that mentioned the JIF, 63% associated it with quality, 40% with impact, importance, or significance, and 20% with prestige, reputation, or status.

In sum, our results show that the use of the JIF is encouraged in RPT evaluations, especially at research-intensive universities, and indicates there is work to be done to improve evaluation processes to avoid the potential misuse of metrics like the JIF.

URL : Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations

DOI : https://doi.org/10.7287/peerj.preprints.27638v2

The Two-Way Street of Open Access Journal Publishing: Flip It and Reverse It

Authors : Lisa Matthias, Najko Jahn, Mikael Laakso

As Open access (OA) is often perceived as the end goal of scholarly publishing, much research has focused on flipping subscription journals to an OA model. Focusing on what can happen after the presumed finish line, this study identifies journals that have converted from OA to a subscription model, and places these “reverse flips” within the greater context of scholarly publishing.

In particular, we examine specific journal descriptors, such as access mode, publisher, subject area, society affiliation, article volume, and citation metrics, to deepen our understanding of reverse flips.

Our results show that at least 152 actively publishing journals have reverse-flipped since 2005, suggesting that this phenomenon does not constitute merely a few marginal outliers, but instead a common pattern within scholarly publishing.

Notably, we found that 62% of reverse flips (N = 95) had not been born-OA journals, but had been founded as subscription journals, and hence have experienced a three-stage transformation from closed to open to closed.

We argue that reverse flips present a unique perspective on OA, and that further research would greatly benefit from enhanced data and tools for identifying such cases.

URL : The Two-Way Street of Open Access Journal Publishing: Flip It and Reverse It 

DOI : https://doi.org/10.3390/publications7020023

Open Science Support as a Portfolio of Services and Projects: From Awareness to Engagement

Authors : Birgit Schmidt, Andrea Bertino, Daniel Beucke, Helene Brinken, Najko Jahn, Lisa Matthias, Julika Mimkes , Katharina Müller, Astrid Orth, Margo Bargheer

Together with many other universities worldwide, the University of Göttingen has aimed to unlock the full potential of networked digital scientific communication by strengthening open access as early as the late 1990s.

Open science policies at the institutional level consequently followed and have been with us for over a decade. However, for several reasons, their adoption often is still far from complete when it comes to the practices of researchers or research groups.

To improve this situation at our university, there is dedicated support at the infrastructural level: the university library collaborates with several campus units in developing and running services, activities and projects in support of open access and open science.

This article outlines our main activity areas and aligns them with the overall rationale to reach higher uptake and acceptance of open science practice at the university. The mentioned examples of our activities highlight how we seek to advance open science along the needs and perspectives of diverse audiences and by running it as a multi-stakeholder endeavor.

Therefore, our activities involve library colleagues with diverse backgrounds, faculty and early career researchers, research managers, as well as project and infrastructure staff. We conclude with a summary of achievements and challenges to be faced.

URL : Open Science Support as a Portfolio of Services and Projects: From Awareness to Engagement

DOI : https://doi.org/10.3390/publications6020027

A multi-disciplinary perspective on emergent and future innovations in peer review

Authors : Jonathan P. Tennant, Jonathan M. Dugan, Daniel Graziotin, Damien C. Jacques, François Waldner, Daniel Mietchen, Yehia Elkhatib, Lauren B. Collister, Christina K. Pikas, Tom Crick, Paola Masuzzo, Anthony Caravaggi, Devin R. Berg, Kyle E. Niemeyer, Tony Ross-Hellauer, Sara Mannheimer, Lillian Rigling, Daniel S. Kat, Bastian Greshake Tzovaras, Josmel Pacheco-Mendoza, Nazeefa Fatima, Marta Poblet, Marios Isaakidis, Dasapta Erwin Irawan, Sébastien Renaut, Christopher R. Madan, Lisa Matthias, Jesper Nørgaard Kjær, Daniel Paul O’Donnell, Cameron Neylon, Sarah Kearns, Manojkumar Selvaraju, Julien Colomb

Peer review of research articles is a core part of our scholarly communication system. In spite of its importance, the status and purpose of peer review is often contested. What is its role in our modern digital research and communications infrastructure?

Does it perform to the high standards with which it is generally regarded? Studies of peer review have shown that it is prone to bias and abuse in numerous dimensions, frequently unreliable, and can fail to detect even fraudulent research.

With the advent of Web technologies, we are now witnessing a phase of innovation and experimentation in our approaches to peer review. These developments prompted us to examine emerging models of peer review from a range of disciplines and venues, and to ask how they might address some of the issues with our current systems of peer review.

We examine the functionality of a range of social Web platforms, and compare these with the traits underlying a viable peer review system: quality control, quantified performance metrics as engagement incentives, and certification and reputation.

Ideally, any new systems will demonstrate that they out-perform current models while avoiding as many of the biases of existing systems as possible. We conclude that there is considerable scope for new peer review initiatives to be developed, each with their own potential issues and advantages.

We also propose a novel hybrid platform model that, at least partially, resolves many of the technical and social issues associated with peer review, and can potentially disrupt the entire scholarly communication system.

Success for any such development relies on reaching a critical threshold of research community engagement with both the process and the platform, and therefore cannot be achieved without a significant change of incentives in research environments.

URL : A multi-disciplinary perspective on emergent and future innovations in peer review

DOI : http://dx.doi.org/10.12688/f1000research.12037.1