Étiquette : Juan Pablo Alperin

Catégories
EN

Science in motion: A qualitative analysis of journalists’ use and perception of preprints

Authors : Alice Fleerackers, Laura L. Moorhead, Lauren A. Maggio, Kaylee Fagan, Juan Pablo Alperin

This qualitative study explores how and why journalists use preprints—unreviewed research papers—in their reporting. Through thematic analysis of interviews conducted with 19 health and science journalists in the second year of the COVID-19 pandemic, it applies a theoretical framework that conceptualizes COVID-19 preprint research as a form of post-normal science, characterized by high scientific uncertainty and societal relevance, urgent need for political decision-making, and value-related policy considerations.

Findings suggest that journalists approach the decision to cover preprints as a careful calculation, in which the potential public benefits and the ease of access preprints provided were weighed against risks of spreading misinformation.

Journalists described viewing unreviewed studies with extra skepticism and relied on diverse strategies to find, vet, and report on them. Some of these strategies represent standard science journalism, while others, such as labeling unreviewed studies as preprints, mark a departure from the norm. However, journalists also reported barriers to covering preprints, as many felt they lacked the expertise or the time required to fully understand or vet the research.

The findings suggest that coverage of preprints is likely to continue post-pandemic, with important implications for scientists, journalists, and the publics who read their work.

URL : Science in motion: A qualitative analysis of journalists’ use and perception of preprints

DOI : https://doi.org/10.1371/journal.pone.0277769

Catégories
EN

How faculty define quality, prestige, and impact in research

Authors : Esteban Morales, Erin McKiernan, Meredith T. Niles, Lesley Schimanski, Juan Pablo Alperin

Despite the calls for change, there is significant consensus that when it comes to evaluating publications, review, promotion, and tenure processes should aim to reward research that is of high “quality,” has an “impact,” and is published in “prestigious” journals.

Nevertheless, such terms are highly subjective and present challenges to ascertain precisely what such research looks like. Accordingly, this article responds to the question: how do faculty from universities in the United States and Canada define the terms quality, prestige, and impact?

We address this question by surveying 338 faculty members from 55 different institutions. This study’s findings highlight that, despite their highly varied definitions, faculty often describe these terms in overlapping ways. Additionally, results shown that marked variance in definitions across faculty does not correspond to demographic characteristics.

This study’s results highlight the need to more clearly implement evaluation regimes that do not rely on ill-defined concepts.

DOI : https://doi.org/10.1101/2021.04.14.439880

Catégories
EN

Communicating Scientific Uncertainty in an Age of COVID-19: An Investigation into the Use of Preprints by Digital Media Outlets

Authors : Alice Fleerackers, Michelle Riedlinger, Laura Moorhead, Rukhsana Ahmed, Juan Pablo Alperin

In this article, we investigate the surge in use of COVID-19-related preprints by media outlets. Journalists are a main source of reliable public health information during crises and, until recently, journalists have been reluctant to cover preprints because of the associated scientific uncertainty.

Yet, uploads of COVID-19 preprints and their uptake by online media have outstripped that of preprints about any other topic. Using an innovative approach combining altmetrics methods with content analysis, we identified a diversity of outlets covering COVID-19-related preprints during the early months of the pandemic, including specialist medical news outlets, traditional news media outlets, and aggregators.

We found a ubiquity of hyperlinks as citations and a multiplicity of framing devices for highlighting the scientific uncertainty associated with COVID-19 preprints. These devices were rarely used consistently (e.g., mentioning that the study was a preprint, unreviewed, preliminary, and/or in need of verification).

About half of the stories we analyzed contained framing devices emphasizing uncertainty. Outlets in our sample were much less likely to identify the research they mentioned as preprint research, compared to identifying it as simply “research.” This work has significant implications for public health communication within the changing media landscape.

While current best practices in public health risk communication promote identifying and promoting trustworthy sources of information, the uptake of preprint research by online media presents new challenges.

At the same time, it provides new opportunities for fostering greater awareness of the scientific uncertainty associated with health research findings.

DOI : https://doi.org/10.1080/10410236.2020.1864892

Catégories
EN

Why we publish where we do: Faculty publishing values and their relationship to review, promotion and tenure expectations

Authors : Meredith T. Niles, Lesley A. Schimanski, Erin C. McKiernan, Juan Pablo Alperin

Using an online survey of academics at 55 randomly selected institutions across the US and Canada, we explore priorities for publishing decisions and their perceived importance within review, promotion, and tenure (RPT).

We find that respondents most value journal readership, while they believe their peers most value prestige and related metrics such as impact factor when submitting their work for publication.

Respondents indicated that total number of publications, number of publications per year, and journal name recognition were the most valued factors in RPT.

Older and tenured respondents (most likely to serve on RPT committees) were less likely to value journal prestige and metrics for publishing, while untenured respondents were more likely to value these factors.

These results suggest disconnects between what academics value versus what they think their peers value, and between the importance of journal prestige and metrics for tenured versus untenured faculty in publishing and RPT perceptions.

URL : Why we publish where we do: Faculty publishing values and their relationship to review, promotion and tenure expectations

DOI : https://doi.org/10.1371/journal.pone.0228914

Catégories
EN

How much research shared on Facebook is hidden from public view? A comparison of public and private online activity around PLOS ONE papers

Authors : Asura Enkhbayar, Stefanie Haustein, Germana Barata, Juan Pablo Alperin

Despite its undisputed position as the biggest social media platform, Facebook has never entered the main stage of altmetrics research. In this study, we argue that the lack of attention by altmetrics researchers is not due to a lack of relevant activity on the platform, but because of the challenges in collecting Facebook data have been limited to activity that takes place in a select group of public pages and groups.

We present a new method of collecting shares, reactions, and comments across the platform-including private timelines-and use it to gather data for all articles published between 2015 to 2017 in the journal PLOS ONE.

We compare the gathered data with altmetrics collected and aggregated by Altmetric. The results show that 58.7% of papers shared on the platform happen outside of public view and that, when collecting all shares, the volume of activity approximates patterns of engagement previously only observed for Twitter.

Both results suggest that the role and impact of Facebook as a medium for science and scholarly communication has been underestimated. Furthermore, they emphasise the importance of openness and transparency around the collection and aggregation of altmetrics.

URL : https://arxiv.org/abs/1909.01476

Catégories
EN

Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations

Authors : Erin C. McKiernan​, Lesley A. Schimanski, Carol Muñoz Nieves, Lisa Matthias, Meredith T. Niles, Juan Pablo Alperin

The Journal Impact Factor (JIF) was originally designed to aid libraries in deciding which journals to index and purchase for their collections. Over the past few decades, however, it has become a relied upon metric used to evaluate research articles based on journal rank. Surveyed faculty often report feeling pressure to publish in journals with high JIFs and mention reliance on the JIF as one problem with current academic evaluation systems.

While faculty reports are useful, information is lacking on how often and in what ways the JIF is currently used for review, promotion, and tenure (RPT). We therefore collected and analyzed RPT documents from a representative sample of 129 universities from the United States and Canada and 381 of their academic units.

We found that 40% of doctoral, research-intensive (R-type) institutions and 18% of master’s, or comprehensive (M-type) institutions explicitly mentioned the JIF, or closely related terms, in their RPT documents.

Undergraduate, or baccalaureate (B-type) institutions did not mention it at all. A detailed reading of these documents suggests that institutions may also be using a variety of terms to indirectly refer to the JIF.

Our qualitative analysis shows that 87% of the institutions that mentioned the JIF supported the metric’s use in at least one of their RPT documents, while 13% of institutions expressed caution about the JIF’s use in evaluations.

None of the RPT documents we analyzed heavily criticized the JIF or prohibited its use in evaluations. Of the institutions that mentioned the JIF, 63% associated it with quality, 40% with impact, importance, or significance, and 20% with prestige, reputation, or status.

In sum, our results show that the use of the JIF is encouraged in RPT evaluations, especially at research-intensive universities, and indicates there is work to be done to improve evaluation processes to avoid the potential misuse of metrics like the JIF.

URL : Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations

DOI : https://doi.org/10.7287/peerj.preprints.27638v2

Catégories
EN

How significant are the public dimensions of faculty work in review, promotion, and tenure documents?

Authors : Juan Pablo Alperin, Gustavo E. Fischman, Erin C. McKiernan, Carol Muñoz Nieves, Meredith T. Niles, Lesley Schimanski

Much of the work of universities, even private institutions, has significant public dimensions. Faculty work in particular is often funded by public funds, is aimed at serving the public good, and is subject to public evaluation.

To understand how the public dimensions of faculty work are valued, we analyzed review, tenure and promotion documents from a representative sample of 129 Canadian and American universities.

We found terms and concepts related to public and community are mentioned in a large portion of documents, but mostly in ways that relate to service—an undervalued aspect of academic careers.

Moreover, we find significant mentions of traditional research outputs and citation-based metrics. Such outputs and metrics reward faculty work targeted to academics, and mostly disregard the public dimensions.

We conclude that institutions that want to live up to their public mission need to work towards systemic change in how faculty work is assessed and incentivized.

URL : How significant are the public dimensions of faculty work in review, promotion, and tenure documents?

DOI : https://hcommons.org/deposits/item/hc:21015