A “basket of metrics”—the best support for understanding journal merit

Authors : Lisa Colledge, Chris James

Aim

To survey opinion of the assertion that useful metricbased input requires a “basket of metrics” to allow more varied and nuanced insights into merit than is possible by using one metric alone.

Methods

A poll was conducted to survey opinions (N=204; average response rate=61%) within the international research community on using usage metrics in merit systems.

Results

“Research is best quantified using multiple criteria” was selected by most (40%) respondents as the reason that usage metrics are valuable, and 95% of respondents indicated that they would be likely or very likely to use usage metrics in their assessments of research merit, if they had access to them.

There was a similar degree of preference for simple and sophisticated usage metrics confirming that one size does not fit all, and that a one-metric approach to merit is insufficient.

Conclusion

This survey demonstrates a clear willingness and a real appetite to use a “basket of metrics” to broaden the ways in which research merit can be detected and demonstrated.

URL : http://europeanscienceediting.eu/articles/a-basket-of-metrics-the-best-support-for-understanding-journal-merit/

Evaluating research and researchers by the journal impact factor: is it better than coin flipping?

Authors : Ricardo Brito, Alonso Rodríguez-Navarro

The journal impact factor (JIF) is the average of the number of citations of the papers published in a journal, calculated according to a specific formula; it is extensively used for the evaluation of research and researchers.

The method assumes that all papers in a journal have the same scientific merit, which is measured by the JIF of the publishing journal. This implies that the number of citations measures scientific merits but the JIF does not evaluate each individual paper by its own number of citations.

Therefore, in the comparative evaluation of two papers, the use of the JIF implies a risk of failure, which occurs when a paper in the journal with the lower JIF is compared to another with fewer citations in the journal with the higher JIF.

To quantify this risk of failure, this study calculates the failure probabilities, taking advantage of the lognormal distribution of citations. In two journals whose JIFs are ten-fold different, the failure probability is low.

However, in most cases when two papers are compared, the JIFs of the journals are not so different. Then, the failure probability can be close to 0.5, which is equivalent to evaluating by coin flipping.

URL : https://arxiv.org/abs/1809.10999

Plurality in multi-disciplinary research: multiple institutional affiliations are associated with increased citations

Authors : Paul Sanfilippo, Alex W. Hewitt, David A. Mackey

Background

The institutional affiliations and associated collaborative networks that scientists foster during their research careers are salient in the production of high-quality science. The phenomenon of multiple institutional affiliations and its relationship to research output remains relatively unexplored in the literature.

Methods

We examined 27,612 scientific articles, modelling the normalized citation counts received against the number of authors and affiliations held.

Results

In agreement with previous research, we found that teamwork is an important factor in high impact papers, with average citations received increasing concordant with the number of co-authors listed.

For articles with more than five co-authors, we noted an increase in average citations received when authors with more than one institutional affiliation contributed to the research.

Discussion

Multiple author affiliations may play a positive role in the production of high-impact science. This increased researcher mobility should be viewed by institutional boards as meritorious in the pursuit of scientific discovery.

URL : Plurality in multi-disciplinary research: multiple institutional affiliations are associated with increased citations

DOI : https://doi.org/10.7717/peerj.5664

Agriculture Journals Covered by Directory of Open Access Journals: An Analytical Study

Author : Muruli Acharya

With the advent of open access movement, open access journals (OAJs) being the prodigious source of academic and research information have been gaining significant magnitude.

The electronic age has made it easier and more convenient than ever to break barriers to research information. The present study aims to study and analyse the status of 497 OAJs in Agriculture indexed in Directory of Open Access Journals.

Specified traits such as Geographic and language wise distribution, coverage of Indexing/Abstracting databases, ranking of journals according to Impact Factor (IF), OA licensing model adopted, policy of plagiarism, visibility on social media and related issues of the OAJs in Agriculture are evaluated in the paper.

Results indicated the dominance of De Gruyter Open as a publisher with highest number of OAJs, English as a content language, Indonesia with highest number of OAJs, Google scholar with highest journals indexed.

The study observes the increasing migration of journals from commercial practice to OA. Frontiers in Plant Science found with highest Impact Factor among OAJs in Agriculture.

URL : Agriculture Journals Covered by Directory of Open Access Journals: An Analytical Study

Alternative location : http://publications.drdo.gov.in/ojs/index.php/djlit/article/view/13114

The Journal Impact Factor: A brief history, critique, and discussion of adverse effects

Authors : Vincent Lariviere, Cassidy R. Sugimoto

The Journal Impact Factor (JIF) is, by far, the most discussed bibliometric indicator. Since its introduction over 40 years ago, it has had enormous effects on the scientific ecosystem: transforming the publishing industry, shaping hiring practices and the allocation of resources, and, as a result, reorienting the research activities and dissemination practices of scholars.

Given both the ubiquity and impact of the indicator, the JIF has been widely dissected and debated by scholars of every disciplinary orientation. Drawing on the existing literature as well as on original research, this chapter provides a brief history of the indicator and highlights well-known limitations-such as the asymmetry between the numerator and the denominator, differences across disciplines, the insufficient citation window, and the skewness of the underlying citation distributions.

The inflation of the JIF and the weakening predictive power is discussed, as well as the adverse effects on the behaviors of individual actors and the research enterprise. Alternative journal-based indicators are described and the chapter concludes with a call for responsible application and a commentary on future developments in journal indicators

URL : https://arxiv.org/abs/1801.08992

Scientist impact factor (SIF): a new metric for improving scientists’ evaluation?

Authors : Giuseppe Lippi, Camilla Mattiuzzi

Background

The publication of scientific research is the mainstay for knowledge dissemination, but is also an essential criterion of scientists’ evaluation for recruiting funds and career progression.

Although the most widespread approach for evaluating scientists is currently based on the H-index, the total impact factor (IF) and the overall number of citations, these metrics are plagued by some well-known drawbacks. Therefore, with the aim to improve the process of scientists’ evaluation, we developed a new and potentially useful indicator of recent scientific output.

Methods

The new metric scientist impact factor (SIF) was calculated as all citations of articles published in the two years following the publication year of the articles, divided by the overall number of articles published in that year. The metrics was then tested by analyzing data of the 40 top scientists of the local University.

Results

No correlation was found between SIF and H-index (r=0.15; P=0.367) or 2 years H-index (r=−0.01; P=0.933), whereas the H-Index and 2 years H-index values were found to be highly correlated (r=0.57; P<0.001). A highly significant correlation was also observed between the articles published in one year and the total number of citations to these articles in the two following years (r=0.62; P<0.001).

Conclusions

According to our data, the SIF may be a useful measure to complement current metrics for evaluating scientific output. Its use may be especially helpful for young scientists, wherein the SIF reflects the scientific output over the past two years thus increasing their chances to apply to and obtain competitive funding.

URL : http://atm.amegroups.com/article/view/15375

What are the personal and professional characteristics that distinguish the researchers who publish in high- and low-impact journals? A multi-national web-based survey

Authors : Carlos Eduardo Paiva, Raphael L C Araujo, Bianca Sakamoto Ribeiro Paiva, Cristiano de Pádua Souza, Flavio Mavignier Cárcano, Marina Moreira Costa, Sérgio Vicente Serrano, João Paulo Nogueira Lima

Purpose

This study identifies the personal and professional profiles of researchers with a greater potential to publish high-impact academic articles.

Method

The study involved conducting an international survey of journal authors using a web-based questionnaire. The survey examined personal characteristics, funding, and the perceived barriers of research quality, work-life balance, and satisfaction and motivation in relation to career.

The processes of manuscript writing and journal publication were measured using an online questionnaire that was developed for this study. The responses were compared between the two groups of researchers using logistic regression models.

Results

A total of 269 questionnaires were analysed. The researchers shared some common perceptions; both groups reported that they were seeking recognition (or to be leaders in their areas) rather than financial remuneration.

Furthermore, both groups identified time and funding constraints as the main obstacles to their scientific activities.

The amount of time that was spent on research activities, having >5 graduate students under supervision, never using text editing services prior to the publication of articles, and living in a developed and English-speaking country were the independent variables that were associated with their article getting a greater chance of publishing in a high-impact journal.

In contrast, using one’s own resources to perform studies decreased the chance of publishing in high-impact journals.

Conclusions

The researchers who publish in high-impact journals have distinct profiles compared with the researchers who publish in low-impact journals.

English language abilities and the actual amount of time that is dedicated to research and scientific writing, as well as aspects that relate to the availability of financial resources are the factors that are associated with a successful researcher’s profile.

URL : What are the personal and professional characteristics that distinguish the researchers who publish in high- and low-impact journals? A multi-national web-based survey

DOI : http://dx.doi.org/10.3332/ecancer.2017.718