Archives des mots-clés : research impact

Readership Data and Research Impact

Posté le par

Authors : Ehsan Mohammadi, Mike Thelwall

Reading academic publications is a key scholarly activity. Scholars accessing and recording academic publications online are producing new types of readership data. These include publisher, repository, and academic social network download statistics as well as online reference manager records.

This chapter discusses the use of download and reference manager data for research evaluation and library collection development. The focus is on the validity and application of readership data as an impact indicator for academic publications across different disciplines.

Mendeley is particularly promising in this regard, although all data sources are not subjected to rigorous quality control and can be manipulated.

URL : https://arxiv.org/abs/1901.08593

Preprints in Scholarly Communication: Re-Imagining Metrics and Infrastructures

Posté le par

Authors : B. Preedip Balaji, M. Dhanamjaya

Digital scholarship and electronic publishing among the scholarly communities are changing when metrics and open infrastructures take centre stage for measuring research impact. In scholarly communication, the growth of preprint repositories over the last three decades as a new model of scholarly publishing has emerged as one of the major developments.

As it unfolds, the landscape of scholarly communication is transitioning, as much is being privatized as it is being made open and towards alternative metrics, such as social media attention, author-level, and article-level metrics. Moreover, the granularity of evaluating research impact through new metrics and social media change the objective standards of evaluating research performance.

Using preprint repositories as a case study, this article situates them in a scholarly web, examining their salient features, benefits, and futures. Towards scholarly web development and publishing on semantic and social web with open infrastructures, citations, and alternative metrics—how preprints advance building web as data is discussed.

We examine that this will viably demonstrate new metrics and in enhancing research publishing tools in scholarly commons facilitating various communities of practice.

However, for the preprint repositories to sustain, scholarly communities and funding agencies should support continued investment in open knowledge, alternative metrics development, and open infrastructures in scholarly publishing.

URL : Preprints in Scholarly Communication: Re-Imagining Metrics and Infrastructures

DOI : https://doi.org/10.3390/publications7010006

On the Heterogeneous Distributions in Paper Citations

Posté le par

Authors : Jinhyuk Yun, Sejung Ahn, June Young Lee

Academic papers have been the protagonists in disseminating expertise. Naturally, paper citation pattern analysis is an efficient and essential way of investigating the knowledge structure of science and technology.

For decades, it has been observed that citation of scientific literature follows a heterogeneous and heavy-tailed distribution, and many of them suggest a power-law distribution, log-normal distribution, and related distributions.

However, many studies are limited to small-scale approaches; therefore, it is hard to generalize. To overcome this problem, we investigate 21 years of citation evolution through a systematic analysis of the entire citation history of 42,423,644 scientific literatures published from 1996 to 2016 and contained in SCOPUS.

We tested six candidate distributions for the scientific literature in three distinct levels of Scimago Journal & Country Rank (SJR) classification scheme. First, we observe that the raw number of annual citation acquisitions tends to follow the log-normal distribution for all disciplines, except for the first year of the publication.

We also find significant disparity between the yearly acquired citation number among the journals, which suggests that it is essential to remove the citation surplus inherited from the prestige of the journals.

Our simple method for separating the citation preference of an individual article from the inherited citation of the journals reveals an unexpected regularity in the normalized annual acquisitions of citations across the entire field of science.

Specifically, the normalized annual citation acquisitions have power-law probability distributions with an exponential cut-off of the exponents around 2.3, regardless of its publication and citation year.

Our results imply that journal reputation has a substantial long-term impact on the citation.

URL : https://arxiv.org/abs/1810.08809

A “basket of metrics”—the best support for understanding journal merit

Posté le par

Authors : Lisa Colledge, Chris James

Aim

To survey opinion of the assertion that useful metricbased input requires a “basket of metrics” to allow more varied and nuanced insights into merit than is possible by using one metric alone.

Methods

A poll was conducted to survey opinions (N=204; average response rate=61%) within the international research community on using usage metrics in merit systems.

Results

“Research is best quantified using multiple criteria” was selected by most (40%) respondents as the reason that usage metrics are valuable, and 95% of respondents indicated that they would be likely or very likely to use usage metrics in their assessments of research merit, if they had access to them.

There was a similar degree of preference for simple and sophisticated usage metrics confirming that one size does not fit all, and that a one-metric approach to merit is insufficient.

Conclusion

This survey demonstrates a clear willingness and a real appetite to use a “basket of metrics” to broaden the ways in which research merit can be detected and demonstrated.

URL : http://europeanscienceediting.eu/articles/a-basket-of-metrics-the-best-support-for-understanding-journal-merit/

Do funding applications where peer reviewers disagree have higher citations? A cross-sectional study

Posté le par

Authors : Adrian G Barnett, Scott R. Glisson, Stephen Gallo

Background

Decisions about which applications to fund are generally based on the mean scores of a panel of peer reviewers. As well as the mean, a large disagreement between peer reviewers may also be worth considering, as it may indicate a high-risk application with a high return.

Methods

We examined the peer reviewers’ scores for 227 funded applications submitted to the American Institute of Biological Sciences between 1999 and 2006. We examined the mean score and two measures of reviewer disagreement: the standard deviation and range.

The outcome variable was the relative citation ratio, which is the number of citations from all publications associated with the application, standardised by field and publication year.

Results

There was a clear increase in relative citations for applications with a better mean. There was no association between relative citations and either of the two measures of disagreement.

Conclusions

We found no evidence that reviewer disagreement was able to identify applications with a higher than average return. However, this is the first study to empirically examine this association, and it would be useful to examine whether reviewer disagreement is associated with research impact in other funding schemes and in larger sample sizes.

URL : Do funding applications where peer reviewers disagree have higher citations? A cross-sectional study

DOI : http://dx.doi.org/10.12688/f1000research.15479.2

How significant are the public dimensions of faculty work in review, promotion, and tenure documents?

Posté le par

Authors : Juan Pablo Alperin, Gustavo E. Fischman, Erin C. McKiernan, Carol Muñoz Nieves, Meredith T. Niles, Lesley Schimanski

Much of the work of universities, even private institutions, has significant public dimensions. Faculty work in particular is often funded by public funds, is aimed at serving the public good, and is subject to public evaluation.

To understand how the public dimensions of faculty work are valued, we analyzed review, tenure and promotion documents from a representative sample of 129 Canadian and American universities.

We found terms and concepts related to public and community are mentioned in a large portion of documents, but mostly in ways that relate to service—an undervalued aspect of academic careers.

Moreover, we find significant mentions of traditional research outputs and citation-based metrics. Such outputs and metrics reward faculty work targeted to academics, and mostly disregard the public dimensions.

We conclude that institutions that want to live up to their public mission need to work towards systemic change in how faculty work is assessed and incentivized.

URL : How significant are the public dimensions of faculty work in review, promotion, and tenure documents?

DOI : https://hcommons.org/deposits/item/hc:21015

Plurality in multi-disciplinary research: multiple institutional affiliations are associated with increased citations

Posté le par

Authors : Paul Sanfilippo​, Alex W. Hewitt, David A. Mackey

Background

The institutional affiliations and associated collaborative networks that scientists foster during their research careers are salient in the production of high-quality science. The phenomenon of multiple institutional affiliations and its relationship to research output remains relatively unexplored in the literature.

Methods

We examined 27,612 scientific articles, modelling the normalized citation counts received against the number of authors and affiliations held.

Results

In agreement with previous research, we found that teamwork is an important factor in high impact papers, with average citations received increasing concordant with the number of co-authors listed.

For articles with more than five co-authors, we noted an increase in average citations received when authors with more than one institutional affiliation contributed to the research.

Discussion

Multiple author affiliations may play a positive role in the production of high-impact science. This increased researcher mobility should be viewed by institutional boards as meritorious in the pursuit of scientific discovery.

URL : Plurality in multi-disciplinary research: multiple institutional affiliations are associated with increased citations

DOI : https://doi.org/10.7717/peerj.5664