Archives des mots-clés : Scientometrics

Le classement de Leiden environnement scientifique et configuration…

Statut

Le classement de Leiden: environnement scientifique et configuration :

“Le classement de Leiden s’impose aujourd’hui comme une alternative pertinente et valable vis-à-vis de celui de Shanghai. De nombreux indicateurs font intervenir les caractéristiques propres aux champs disciplinaires et des calculs fondés sur le principe de distribution. Il est conçu par le centre CWTS de l’université néerlandaise de Leiden.”

“The Leiden Ranking is considered today as quite a pertinent and valuable alternative vs. the Shanghai Ranking. A significant number of indicators involve for instance Fields Citation Scores and data distribution. It is conceived by the CWTS of the University of Leiden – The Netherlands.”

URL : http://archivesic.ccsd.cnrs.fr/sic_00696098

Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact

Posté le par

Background

Citations in peer-reviewed articles and the impact factor are generally accepted measures of scientific impact. Web 2.0 tools such as Twitter, blogs or social bookmarking tools provide the possibility to construct innovative article-level or journal-level metrics to gauge impact and influence. However, the relationship of the these new metrics to traditional metrics such as citations is not known.

Objective:

(1) To explore the feasibility of measuring social impact of and public attention to scholarly articles by analyzing buzz in social media, (2) to explore the dynamics, content, and timing of tweets relative to the publication of a scholarly article, and (3) to explore whether these metrics are sensitive and specific enough to predict highly cited articles.

Methods

Between July 2008 and November 2011, all tweets containing links to articles in the Journal of Medical Internet Research (JMIR) were mined. For a subset of 1573 tweets about 55 articles published between issues 3/2009 and 2/2010, different metrics of social media impact were calculated and compared against subsequent citation data from Scopus and Google Scholar 17 to 29 months later. A heuristic to predict the top-cited articles in each issue through tweet metrics was validated.

Results

A total of 4208 tweets cited 286 distinct JMIR articles. The distribution of tweets over the first 30 days after article publication followed a power law (Zipf, Bradford, or Pareto distribution), with most tweets sent on the day when an article was published (1458/3318, 43.94% of all tweets in a 60-day period) or on the following day (528/3318, 15.9%), followed by a rapid decay. The Pearson correlations between tweetations and citations were moderate and statistically significant, with correlation coefficients ranging from .42 to .72 for the log-transformed Google Scholar citations, but were less clear for Scopus citations and rank correlations. A linear multivariate model with time and tweets as significant predictors (P < .001) could explain 27% of the variation of citations. Highly tweeted articles were 11 times more likely to be highly cited than less-tweeted articles (9/12 or 75% of highly tweeted article were highly cited, while only 3/43 or 7% of less-tweeted articles were highly cited; rate ratio 0.75/0.07 = 10.75, 95% confidence interval, 3.4–33.6). Top-cited articles can be predicted from top-tweeted articles with 93% specificity and 75% sensitivity.

Conclusions

Tweets can predict highly cited articles within the first 3 days of article publication. Social media activity either increases citations or reflects the underlying qualities of the article that also predict citations, but the true use of these metrics is to measure the distinct concept of social impact. Social impact measures based on tweets are proposed to complement traditional citation metrics. The proposed twimpact factor may be a useful and timely metric to measure uptake of research findings and to filter research findings resonating with the public in real time.

URL : http://www.jmir.org/2011/4/e123/

L’évaluation des publications scientifiques : nouvelles approches, nouveaux enjeux

Posté le par

Après avoir resitué l’évaluation dans un contexte international (notamment Shanghai) et national (programme P150 de la LOLF), l’étude porte sur une analyse critique du modèle ISI et des modèles alternatifs (Eigenfactor et dérivés du Weighted Page Rank). Elle tente de définir les conditions d’une évaluation plus qualitative en lien avec les Archives ouvertes.

URL : http://archivesic.ccsd.cnrs.fr/sic_00589641/fr/

Le JCR facteur d’impact (IF) et le SCImago Journal Rank Indicator (SJR) des revues françaises : une étude comparative

Posté le par

Auteurs/Authors : Joachim Schöpfel, Hélène Prost

Une des fonctions principales des revues scientifiques est de contribuer à l’évaluation de la recherche et des chercheurs. Depuis plus de 50 ans, le facteur d’impact (IF) de l’Institute of Scientific Information (ISI) est devenu l’indicateur dominant de la qualité d’une revue, malgré certaines faiblesses et critiques dont notamment la sur-représentation des revues anglophones. Cela est un handicap aussi bien pour les chercheurs français que pour les éditeurs francophones ; publier en français n’est pas valorisant.

Or, il existe depuis 2007 une alternative sérieuse à l’IF : le nouveau SCImago Journal Rank Indicator (SJR) qui applique l’algorithme de Google (PageRank) aux revues de la base bibliographique SCOPUS dont la couverture est plus large que celle de l’ISI.

Le but de notre étude est de comparer ces deux indicateurs par rapport aux titres français. L’objectif est de répondre à trois questions : Quelle est la couverture pour les titres français indexés par l’ISI et par SCOPUS (nombre de revues, domaines scientifiques) ? Quelles sont les différences des deux indicateurs IF et SJR par rapport aux revues françaises (classement) ? Quel est l’intérêt du SJR pour l’évaluation, en termes de représentativité des titres français ?

Les résultats de notre analyse de 368 revues françaises avec IF et/ou SJR sont plutôt encourageants pour une utilisation du nouvel indicateur SJR, du moins en complémentarité au IF :

(1) Couverture : 166 revues sont indexées par l’ISI (45 %), 345 revues par SCOPUS (94 %), 143 revues par les deux (39 %). 82% des revues sont issus des domaines STM, 18% des domaines SHS. La couverture de SCOPUS est meilleure surtout en médecine et pharmacologie.

(2) Classement : Pour les titres avec IF et SJR, la corrélation entre les deux indicateurs est significative (0,76). En termes de classement (ranking), l’IF différencie mieux les revues que le SJR (155 vs. 89 rangs). En revanche, du fait de la couverture plus exhaustive de SCOPUS, le SJR rend visible au niveau international davantage de titres.

(3) Représentativité : L’intérêt de SCOPUS et du SJR réside dans la couverture plus représentative de l’édition française (19% vs 9% pour ISI/IF), notamment en STM (38% vs 19 %), beaucoup moins en SHS (6% vs 2 %). Sont indexés surtout les titres de quelques grands éditeurs français ou internationaux ; la plupart des éditeurs français (80 %–90 %) n’ont aucun titre dans le JCR et/ou SCOPUS, même si de nouveau SCOPUS est plus représentatif (avec 17% des éditeurs vs 10% pour le JCR).

Les problèmes méthodologiques et les perspectives pour une évaluation multidimensionnelle sont discutés. L’étude compare le IF et le SJR par rapport aux 368 titres français avec IF et/ou SJR. Les résultats : La couverture du SJR est plus large que celle de l’IF (94% vs 45%) et meilleure surtout dans les sciences médicales. Pour les titres avec IF et SJR, la corrélation entre les deux indicateurs est significative (0,76). En termes de classement (ranking), l’IF différencie mieux les revues que le SJR (155 vs 89 rangs). L’intérêt du SJR réside dans la couverture plus représentative de l’édition française (19% vs 9% avec IF), notamment en STM (38% vs 19 %), moins en SHS (6% vs 2 %).

URL : http://archivesic.ccsd.cnrs.fr/sic_00567847/fr/

Multivariate approach to classify resear…

Posté le par

Multivariate approach to classify research institutes according to their outputs: the case of the CSIC’s institutes :

“This paper attempts to build a classification model according to the research products created by those institutes and hence to design specific evaluation processes. Several scientific input/output indicators belonging to 109 research institutes from the Spanish National Research Council (CSIC) were selected. A multidimensional approach was proposed to resume these indicators in various components. A clustering analysis was used to classify the institutes according to their scores with those components (principal component analysis). Moreover, the validity of the a priori classification was tested and the most discriminant variables were detected (linear discriminant analysis). Results show that there are three types of institutes according to their research outputs: Humanistic, Scientific and Technological. It is argue that these differences oblige to design more precise assessment exercises which focus on the particular results of each type of institute. We conclude that this method permits to build more precise research assessment exercises which consider the varied nature of the scientific activity.”

URL : http://eprints.rclis.org/handle/10760/15364

Fractional counting of citations in rese…

Posté le par

Fractional counting of citations in research evaluation: An option for cross- and interdisciplinary assessments :

“In the case of the scientometric evaluation of multi- or interdisciplinary units one risks to compare apples with oranges: each paper has to assessed in comparison to an appropriate reference set. We suggest that the set of citing papers first can be considered as the relevant representation of the field of impact. In order to normalize for differences in citation behavior among fields, citations can be fractionally counted proportionately to the length of the reference lists in the citing papers. This new method enables us to compare among units with different disciplinary affiliations at the paper level and also to assess the statistical significance of differences among sets. Twenty-seven departments of the Tsinghua University in Beijing are thus compared. Among them, the Department of Chinese Language and Linguistics is upgraded from the 19th to the second position in the ranking. The overall impact of 19 of the 27 departments is not significantly different at the 5% level when thus normalized for different citation potentials”.

URL : http://arxiv.org/abs/1012.0359

OpenAccess Statistics: Alternative Impac…

Posté le par

OpenAccess Statistics: Alternative Impact Measures for Open Access documents? An examination how to generate interoperable usage information from distributed Open Access services :

“Publishing and bibliometric indicators are of utmost relevance for scientists and research institutions as the impact or importance of a publication (or even of a scientist or an institution) is mostly regarded to be equivalent to a citation-based indicator, e.g. in form of the Journal Impact Factor or the Hirsch-Index. Both on an individual and an institutional level performance measurement depends strongly on these impact scores. This contribution shows that most common methods to assess the impact of scientific publications often discriminate Open Access publications – and by that reduce the attractiveness of Open Access for scientists. Assuming that the motivation to use Open Access publishing services (e.g. a journal or a repository) would increase if these services would convey some sort of reputation or impact to the scientists, alternative models of impact are discussed. Prevailing research results indicate that alternative metrics based on usage information of electronic documents are suitable to complement or to relativize citation-based indicators. Furthermore an insight into the project OpenAccess- Statistics OA-S is given. OA-S implemented an infrastructure to collect document-related usage information from distributed Open Access Repositories in an aggregator service in order to generate interoperable document access information according to three standards (COUNTER, LogEc and IFABC). The service also guarantees the deduplication of users and identical documents on different servers. In a second phase it is not only planned to implement added services like recommender.”

URL : http://eprints.rclis.org/19068/