Usage Bibliometrics as a Tool to Measure Research Activity

Authors : Edwin A. Henneken, Michael J. Kurtz

Measures for research activity and impact have become an integral ingredient in the assessment of a wide range of entities (individual researchers, organizations, instruments, regions, disciplines).

Traditional bibliometric indicators, like publication and citation based indicators, provide an essential part of this picture, but cannot describe the complete picture.

Since reading scholarly publications is an essential part of the research life cycle, it is only natural to introduce measures for this activity in attempts to quantify the efficiency, productivity and impact of an entity.

Citations and reads are significantly different signals, so taken together, they provide a more complete picture of research activity. Most scholarly publications are now accessed online, making the study of reads and their patterns possible.

Click-stream logs allow us to follow information access by the entire research community, real-time. Publication and citation datasets just reflect activity by authors. In addition, download statistics will help us identify publications with significant impact, but which do not attract many citations.

Click-stream signals are arguably more complex than, say, citation signals. For one, they are a superposition of different classes of readers. Systematic downloads by crawlers also contaminate the signal, as does browsing behavior.

We discuss the complexities associated with clickstream data and how, with proper filtering, statistically significant relations and conclusions can be inferred from download statistics.

We describe how download statistics can be used to describe research activity at different levels of aggregation, ranging from organizations to countries. These statistics show a correlation with socio-economic indicators.

A comparison will be made with traditional bibliometric indicators. We will argue that astronomy is representative of more general trends.

URL : https://arxiv.org/abs/1706.02153

Interroger le texte scientifique

Auteur/Author : Guillaume Cabanac

Les documents textuels sont des vecteurs d’information familiers et incontournables de notre société de l’information. Avec l’essor des plateformes numériques et des médias sociaux, le texte se décline désormais en pages web, billets de blogs, commentaires, tweets et tags, entre autres. Auparavant consommateurs passifs, les lecteurs se muent à leur tour en producteurs de contenus.

En résultent des échanges interpersonnels qui tissent des réseaux sociaux numériques s’étendant bien au-delà de nos cercles relationnels. Dans ce contexte, nature et format des textes, intentions de leurs auteurs (informer, rediffuser, critiquer, compléter, corriger, etc.), contexte spatio-temporel ainsi que véracité et fraîcheur variables des informations sont autant de subtilités à intégrer dans les modèles de recherche d’information.

La première partie de ce mémoire présente une synthèse de résultats en recherche d’information visant à modéliser ces facteurs pour améliorer la pertinence des recherches sur des corpus textuels, notamment issus de médias sociaux.

Le programme de recherche que je développe vise également à « interroger le texte » pour révéler des informations au sujet de son contenu, de ses auteurs et de ses lecteurs. Le texte scientifique a été choisi comme cible pour la richesse de son contenu et de ses méta- données. Ainsi, la deuxième partie du mémoire synthétise des résultats en scientométrie, terme désignant l’étude quantitative des sciences et de l’innovation.

Il s’est agi de questionner des textes scientifiques et les réseaux sous-jacents (lexique, références, auteurs, institutions, etc.) pour faire émerger des connaissances à forte valeur ajoutée et apporter un éclairage sur la création et la diffusion des savoirs scientifiques.

Les deux volets articulés dans ce mémoire concourent à définir un programme de recherche interdisciplinaire à la croisée de l’informatique, la scientométrie et la sociologie des sciences.

Son ambition consiste à interroger le texte scientifique pour en améliorer l’accès (via la recherche d’information) tout en contribuant à éliciter les ressorts de la genèse et de l’évolution des mondes sociaux et des savoirs en sciences (via la scientométrie).

URL : Interroger le texte scientifique

Alternative location : https://tel.archives-ouvertes.fr/tel-01413878/

Bibliometrics and academic staff assessment in Polish university libraries – current trends

Authors : Danuta Ryś, Anna Chadaj

Academic staff assessment in Poland is, to a large extent, based on bibliographic indicators, such as the number of scientific publications produced, the Ministry of Science and Higher Education score pertaining to the journal rank and the publication type, as well as the number of citations and derivatives.

Relevant data is retrieved from bibliographic databases developed by libraries, international citation indexes available for Polish scientific institutions under a national licence, and from open-access international and Polish sources, which are briefly presented in the article.

The workload entailed, and in consequence, the results of this citation search vary depending on the search method applied. For this reason university staff members and university authorities often seek assistance for this from the university library staff. This in return provides an opportunity for libraries to increase their role within the academic community.

In order to investigate the matter further, the authors conducted a survey among the largest academic libraries in Poland.

The findings confirm that bibliometric processes (namely, the registration and the formal acceptance of university staff scientific publications, and compilation of citation reports) have become a vital part of modern library work. Bibliographies of university staff publications developed by libraries include various bibliometric indicators (those most frequently used being identified in the article), and have become an important source of statistical and bibliometric information.

The survey results highlight the most frequently used bibliometric sources and methods. Examples of bibliographic databases created by the libraries and bibliometric indicators used within these databases are also presented.

DOI : https://www.liberquarterly.eu/article/10.18352/lq.10175/

Characterization, description, and considerations for the use of funding acknowledgement data in Web of Science

Funding acknowledgements found in scientific publications have been used to study the impact of funding on research since the 1970s. However, no broad scale indexation of that paratextual element was done until 2008, when Thomson Reuters Web of Science started to add funding acknowledgement information to its bibliographic records.

As this new information provides a new dimension to bibliometric data that can be systematically exploited, it is important to understand the characteristics of these data and the underlying implications for their use.

This paper analyses the presence and distribution of funding acknowledgement data covered in Web of Science.

Our results show that prior to 2009 funding acknowledgements coverage is extremely low and therefore not reliable. Since 2008, funding information has been collected mainly for publications indexed in the Science Citation Index Expanded (SCIE); more recently (2015), inclusion of funding texts for publications indexed in the Social Science Citation Index (SSCI) has been implemented.

Arts & Humanities Citation Index (AHCI) content is not indexed for funding acknowledgement data. Moreover, English-language publications are the most reliably covered.

Finally, not all types of documents are equally covered for funding information indexation and only articles and reviews show consistent coverage.

The characterization of the funding acknowledgement information collected by Thomson Reuters can therefore help understand the possibilities offered by the data but also their limitations.

URL : http://arxiv.org/abs/1604.04780

The application of bibliometrics to research evaluation in the humanities and social sciences: an exploratory study using normalized Google Scholar data for the publications of a research institute

In the humanities and social sciences, bibliometric methods for the assessment of research performance are (so far) less common. The current study takes a concrete example in an attempt to evaluate a research institute from the area of social sciences and humanities with the help of data from Google Scholar (GS).

In order to use GS for a bibliometric study, we have developed procedures for the normalisation of citation impact, building on the procedures of classical bibliometrics. In order to test the convergent validity of the normalized citation impact scores, we have calculated normalized scores for a subset of the publications based on data from the WoS or Scopus.

Even if scores calculated with the help of GS and WoS/Scopus are not identical for the different publication types (considered here), they are so similar that they result in the same assessment of the institute investigated in this study: For example, the institute’s papers whose journals are covered in WoS are cited at about an average rate (compared with the other papers in the journals).

URL :  : https://figshare.com/articles/The_application_of_bibliometrics_to_research_evaluation_in_the_humanities_and_social_sciences_an_exploratory_study_using_normalized_Google_Scholar_data_for_the_publications_of_a_research_institute/1293588

Meaningful Metrics: A 21st Century Librarian’s Guide to Bibliometrics, Altmetrics, and Research Impact

What does it mean to have meaningful metrics in today’s complex higher education landscape? With a foreword by Heather Piwowar and Jason Priem, this highly engaging and activity-laden book serves to introduce readers to the fast-paced world of research metrics from the unique perspective of academic librarians and LIS practitioners.

Starting with the essential histories of bibliometrics and altmetrics, and continuing with in-depth descriptions of the core tools and emerging issues at stake in the future of both fields, Meaningful Metrics is a convenient all-in-one resource that is designed to be used by a range of readers, from those with little to no background on the subject to those looking to become movers and shakers in the current scholarly metrics movement. Authors Borchardt and Roemer, offer tips, tricks, and real-world examples illustrate how librarians can support the successful adoption of research metrics, whether in their institutions or across academia as a whole.

URL : http://www.ala.org/acrl/sites/ala.org.acrl/files/content/publications/booksanddigitalresources/digital/9780838987568_metrics_OA.pdf

Coauthorship networks: A directed network approach considering the order and number of coauthors

“In many scientific fields, the order of coauthors on a paper conveys information about each individual’s contribution to a piece of joint work. We argue that in prior network analyses of coauthorship networks, the information on ordering has been insufficiently considered because ties between authors are typically symmetrized. This is basically the same as assuming that each co-author has contributed equally to a paper. We introduce a solution to this problem by adopting a coauthorship credit allocation model proposed by Kim and Diesner (2014), which in its core conceptualizes co-authoring as a directed, weighted, and self-looped network. We test and validate our application of the adopted framework based on a sample data of 861 authors who have published in the journal Psychometrika. Results suggest that this novel sociometric approach can complement traditional measures based on undirected networks and expand insights into coauthoring patterns such as the hierarchy of collaboration among scholars. As another form of validation, we also show how our approach accurately detects prominent scholars in the Psychometric Society affiliated with the journal.”

URL : http://arxiv.org/abs/1503.00361