Novel processes and metrics for a scientific evaluation rooted in the principles of science

Posté le par

Authors : Michaël Bon, Michael Taylor, Gary S McDowell

Scientific evaluation is a determinant of how scientists, institutions and funders behave, and as such is a key element in the making of science. In this article, we propose an alternative to the current norm of evaluating research with journal rank.

Following a well-defined notion of scientific value, we introduce qualitative processes that can also be quantified and give rise to meaningful and easy-to-use article-level metrics.

In our approach, the goal of a scientist is transformed from convincing an editorial board through a vertical process to convincing peers through an horizontal one. We argue that such an evaluation system naturally provides the incentives and logic needed to constantly promote quality, reproducibility, openness and collaboration in science.

The system is legally and technically feasible and can gradually lead to the self-organized reappropriation of the scientific process by the scholarly community and its institutions. We propose an implementation of our evaluation system with the platform “the Self-Journals of Science” (www.sjscience.org).

URL : Novel processes and metrics for a scientific evaluation rooted in the principles of science

Alternative location : http://www.sjscience.org/article?id=580

A metadata-driven approach to data repository design

Posté le par

Authors : Matthew J. Harvey, Andrew McLean, Henry S. Rzepa

 The design and use of a metadata-driven data repository for research data management is described. Metadata is collected automatically during the submission process whenever possible and is registered with DataCite in accordance with their current metadata schema, in exchange for a persistent digital object identifier.

Two examples of data preview are illustrated, including the demonstration of a method for integration with commercial software that confers rich domain-specific data analytics without introducing customisation into the repository itself.

URL : A metadata-driven approach to data repository design

DOI : http://jcheminf.springeropen.com/articles/10.1186/s13321-017-0190-6

E-Journals and the Big Deal: A Review of the Literature

Posté le par

Author : Cindy Sjoberg

Faced with shrinking budgets and increased subscription prices, many academic libraries are seeking ways to reduce the cost of e-journal access. A common target for cuts is the “Big Deal,” or large bundled subscription model, a term coined by Kenneth Frazier in a 2001 paper criticizing the effects of the Big Deal on the academic community.

The purpose of this literature review is to examine issues related to reducing e-journal costs, including criteria for subscription retention or cancellation, decision-making strategies, impacts of cancellations, and other options for e-journal content provision. Commonly used criteria for decision-making include usage statistics, overlap analysis, and input from subject specialists.

The most commonly used strategy for guiding the process and aggregating data is the rubric or decision grid. While the e-journal landscape supports several access models, such as Pay-Per-View, cloud access, and interlibrary loan, the Big Deal continues to dominate. Trends over the past several years point to dwindling support for the Big Deal however, due largely to significant annual rate increases and loss of content control.

URL : http://scholarworks.sjsu.edu/slissrj/vol6/iss2/3/

Patent citation data in social science research: Overview and best practices

Posté le par

Authors : Adam B. Jaffe, Gaétan de Rassenfosse

The last 2 decades have witnessed a dramatic increase in the use of patent citation data in social science research. Facilitated by digitization of the patent data and increasing computing power, a community of practice has grown up that has developed methods for using these data to: measure attributes of innovations such as impact and originality; to trace flows of knowledge across individuals, institutions and regions; and to map innovation networks.

The objective of this article is threefold. First, it takes stock of these main uses. Second, it discusses 4 pitfalls associated with patent citation data, related to office, time and technology, examiner, and strategic effects. Third, it highlights gaps in our understanding and offers directions for future research.

URL : Patent citation data in social science research: Overview and best practices

Alternative location : http://onlinelibrary.wiley.com/doi/10.1002/asi.23731/full

Développer des ressources numériques pour un public spécialisé : enjeux bibliothéconomiques et informationnels

Posté le par

Auteur/Author : Camille Hervé

Le centre de documentation de l’Institut Bioforce ambitionne de développer ses ressources numériques avec notamment un service de prêt numérique. Cette mission de stage a abouti à l’élaboration d’un plan d’action suivant une méthodologie de projet appuyée par des concepts de bibliothéconomie et des sciences de l’information et de la communication.

Ceci a permis de s’intéresser à la notion de document numérique et ses enjeux dans un contexte organisationnel spécifique, grâce à une analyse des sources de documentation numérique.

URL : Développer des ressources numériques pour un public spécialisé : enjeux bibliothéconomiques et informationnels

Alternative location : http://www.enssib.fr/bibliotheque-numerique/notices/67252-developper-des-ressources-numeriques-pour-un-public-specialise-enjeux-bibliotheconomiques-et-informationnels

Science and Facebook: the same popularity law!

Posté le par

Authors : Zoltán Néda, Levente Varga, Tamás S. Biró

The distribution of scientific citations for publications selected with different rules (author, topic, institution, country, journal, etc.) collapse on a single curve if one plots the citations relative to their mean value.

We find that the distribution of shares for the Facebook posts re-scale in the same manner to the very same curve with scientific citations. This finding suggests that citations are subjected to the same growth mechanism with Facebook popularity measures, being influenced by a statistically similar social environment and selection mechanism.

In a simple master-equation approach the exponential growth of the number of publications and a preferential selection mechanism leads to a Tsallis-Pareto distribution offering an excellent description for the observed statistics.

Based on our model and on the data derived from PubMed we predict that according to the present trend the average citations per scientific publications exponentially relaxes to about 4.

URL : https://arxiv.org/abs/1701.05347

Quantifying the distribution of editorial power and manuscript decision bias at the mega-journal PLOS ONE

Posté le par

Author : Alexander M. Petersen

We analyzed the longitudinal activity of nearly 7,000 editors at the mega-journal PLOS ONE over the 10-year period 2006-2015. Using the article-editor associations, we develop editor-specific measures of power, activity, article acceptance time, citation impact, and editorial renumeration (an analogue to self-citation).

We observe remarkably high levels of power inequality among the PLOS ONE editors, with the top-10 editors responsible for 3,366 articles — corresponding to 2.4% of the 141,986 articles we analyzed. Such high inequality levels suggest the presence of unintended incentives, which may reinforce unethical behavior in the form of decision-level biases at the editorial level.

Our results indicate that editors may become apathetic in judging the quality of articles and susceptible to modes of power-driven misconduct. We used the longitudinal dimension of editor activity to develop two panel regression models which test and verify the presence of editor-level bias.

In the first model we analyzed the citation impact of articles, and in the second model we modeled the decision time between an article being submitted and ultimately accepted by the editor.

We focused on two variables that represent social factors that capture potential conflicts-of-interest: (i) we accounted for the social ties between editors and authors by developing a measure of repeat authorship among an editor’s article set, and (ii) we accounted for the rate of citations directed towards the editor’s own publications in the reference list of each article he/she oversaw.

Our results indicate that these two factors play a significant role in the editorial decision process. Moreover, these two effects appear to increase with editor age, which is consistent with behavioral studies concerning the evolution of misbehavior and response to temptation in power-driven environments.

URL : https://arxiv.org/abs/1701.04906