The application of bibliometrics to research evaluation in the humanities and social sciences: an exploratory study using normalized Google Scholar data for the publications of a research institute

In the humanities and social sciences, bibliometric methods for the assessment of research performance are (so far) less common. The current study takes a concrete example in an attempt to evaluate a research institute from the area of social sciences and humanities with the help of data from Google Scholar (GS).

In order to use GS for a bibliometric study, we have developed procedures for the normalisation of citation impact, building on the procedures of classical bibliometrics. In order to test the convergent validity of the normalized citation impact scores, we have calculated normalized scores for a subset of the publications based on data from the WoS or Scopus.

Even if scores calculated with the help of GS and WoS/Scopus are not identical for the different publication types (considered here), they are so similar that they result in the same assessment of the institute investigated in this study: For example, the institute’s papers whose journals are covered in WoS are cited at about an average rate (compared with the other papers in the journals).

URL :  : https://figshare.com/articles/The_application_of_bibliometrics_to_research_evaluation_in_the_humanities_and_social_sciences_an_exploratory_study_using_normalized_Google_Scholar_data_for_the_publications_of_a_research_institute/1293588

L’édition scientifique institutionnelle en France : État des lieux, matière à réflexions, recommandations

Le présent volume constitue les résultats de l’enquête diligentée par la Direction générale pour la recherche et l’innovation du ministère de l’Enseignement supérieur et de la Recherche et confiée à l’Association des éditeurs de la Recherche et de l’Enseignement supérieur (AEDRES).

L’objectif de la mission confiée à notre association était essentiellement de dresser un état des lieux des activités éditoriales des établissements d’enseignement supérieur et de recherche, d’apporter des informations sur les relations entre édition publique et édition privée et de formuler des propositions pour améliorer ces relations et dynamiser les politiques de valorisation de la recherche.

Le contenu de ce document est important à plusieurs titres.

Tout d’abord parce que les outils statistiques ou d’analyse manquent pour mener une réflexion approfondie sur l’édition scientifique institutionnelle, et certains se prennent, à loisir, à rappeler sa diversité, son opacité, allant parfois jusqu’à douter de son efficacité, voire à remettre en cause sa légitimité.

Cette publication intervient à un moment où les structures françaises de l’enseignement supérieur et de la recherche connaissent de profondes mutations avec d’inévitables conséquences sur leurs structures éditoriales. La mise en place des COmUE devrait ainsi conduire les établissements à repenser la mission de diffusion de la culture et de l’information scientifique et technique par l’édition et la commercialisation d’ouvrages et de périodiques telle qu’elle leur a été confiée par la loi du 26 janvier 1984 (loi Savary).

La question aujourd’hui n’est pas tant celle de la frontière mouvante entre le secteur marchand et le secteur non marchand : les difficultés budgétaires et certaines politiques d’établissement conduisent les presses d’université à chercher plus que jamais à capter le « grand public cultivé » – susceptible de lui apporter quelques assurances sur le plan économique – tandis que l’édition privée opte de plus en plus pour des participations, financière ou matérielle, des chercheurs ou de leurs centres de recherche. De nouveaux modèles sont sans doute à développer ou à inventer.

Par ailleurs, l’édition scientifique est elle-même à un tournant de son histoire avec l’émergence de nouveaux modèles de diffusion qui vont contraindre les professionnels – publics ou privés – à modifier leurs structures et leurs modes de fonctionnement.

Mais que l’on ne s’y trompe pas, quel que soit le support choisi, édition papier ou édition numérique, que l’on penche pour l’« Open Access » ou l’édition payante, le travail d’expertise et de sélection réalisé par les éditeurs de l’enseignement supérieur est absolument nécessaire. Que l’on choisisse une diffusion « élargie » par l’intermédiaire de la filière de la librairie ou que l’on se limite à un public de chercheurs par le biais des bibliothèques et centres de documentation, les garanties offertes par le filtre des éditions universitaires sont indispensables pour assurer la qualité scientifique des publications et le respect de la propriété intellectuelle des auteurs et on peut l’espérer – pour se prémunir contre le plagiat.

Le constat effectué dans cette étude montre combien les presses universitaires sont d’ores et déjà conscientes des enjeux. Elles ont développé des outils, certes perfectibles, qui prennent en compte les nouvelles données : réseaux de diffusion transdisciplinaires, portail de diffusion, modèles économiques et comptables rationalisés, normes d’évaluation scientifique des tapuscrits…

Le bilan dressé comme les recommandations proposées dans les pages qui suivent nous paraissent essentiels pour nourrir une réflexion qui doit être menée conjointement par tous les acteurs de la chaîne du livre – organisations professionnelles des éditeurs publics ou privés, libraires, bibliothécaires, diffuseurs, distributeurs… – mais aussi tous les intervenants de la recherche scientifique : chercheurs, étudiants, directeurs d’établissement et leur personnel, instances de tutelle et d’évaluation…

URL : http://www.allianceathena.fr/sites/default/files/Rapport_AEDRES_EdScientifique_JMHenny.pdf

Improving the peer-review process and editorial quality: key errors escaping the review and editorial process in top scientific journals

We apply a novel mistake index to assess trends in the proportion of corrections published between 1993 and 2014 in Nature, Science and PNAS. The index revealed a progressive increase in the proportion of corrections published in these three high-quality journals.

The index appears to be independent of the journal impact factor or the number of items published, as suggested by a comparative analyses among 16 top scientific journals of different impact factors and disciplines. A more detailed analysis suggests that the trend in the time-to-correction increased significantly over time and also differed among journals (Nature 233 days; Science 136 days; PNAS 232 days).

A detailed review of 1,428 errors showed that 60% of corrections were related to figures, authors, references or results. According to the three categories established, 34.7% of the corrections were considered mild, 47.7% moderate and 17.6% severe, also differing among journals. Errors occurring during the printing process were responsible for 5% of corrections in Nature, 3% in Science and 18% in PNAS.

The measurement of the temporal trends in the quality of scientific manuscripts can assist editors and reviewers in identifying the most common mistakes, increasing the rigor of peer-review and improving the quality of published scientific manuscripts.

URL : Improving the peer-review process and editorial quality: key errors escaping the review and editorial process in top scientific journals

DOI : https://doi.org/10.7717/peerj.1670

Principles of the Self Journal of Science: bringing ethics and freedom to scientific publishing

I present the core principles of the “Self-Journal of Science” (SJS), an open repository as well as a new paradigm of scientific publication. Rooted in Science ethics, a full and consistent solution is proposed to address the many flaws in current systems. SJS implements an optimal peer review, which itself becomes a measurable process, and builds an objective and unfalsifiable evaluation system.

In addition, it can operate at very low costs. One of the essential features of SJS is to allow every scientist to play his full role as a member of the scientific community and to be credited for all contributions – whether as author, referee, or editor. The output is the responsibility of each scientist, and no subgroup can dictate scientific policy to all.

By fully opening up the process of publication, peer pressure becomes the force that drives output towards the highest quality in a virtuous self-regulating circle. SJS also provides a self-organizing and scalable solution to handle an ever-increasing number of articles.

URL : Principles of the Self Journal of Science: bringing ethics and freedom to scientific publishing

Alternative location : http://www.sjscience.org/article?id=46

Peer review in megajournals compared with traditional scholarly journals: Does it make a difference?

A megajournal is an open-access journal that publishes any manuscript that presents scientifically trustworthy empirical results, without asking about the potential scientific contribution prior to publication. Megajournals have rapidly increased their output and are currently publishing around 50,000 articles per year.

We report on a small pilot study in which we looked at the citation distributions for articles in megajournals compared with journals with traditional peer review, which also evaluate articles for contribution and novelty.We found that elite journals with very low acceptance rates have far fewer articles with no or few citations, but that the long tail of articles with two citations or less was actually bigger in a sample of selective traditional journals in comparison with megajournals.

This indicates the need for more systematic studies, because the results raise many questions as to how efficiently the current peer review system in reality fulfils its filtering function.

URL : http://onlinelibrary.wiley.com/doi/10.1002/leap.1007/abstract

Leading by Example? ALA Division Publications, Open Access, and Sustainability

This investigation explores scholarly communication business models in American Library Association (ALA) division peer-reviewed academic journals. Previous studies reveal the numerous issues organizations and publishers face in the academic publishing environment. Through an analysis of documented procedures, policies, and finances of five ALA division journals, we compare business and access models.

We conclude that some ALA divisions prioritize the costs associated with changing business models, including hard-to-estimate costs such as the labor of volunteers. For other divisions, the financial aspects are less important than maintaining core values, such as those defined in ALA’s Core Values in Librarianship.

URL : http://m.crl.acrl.org/content/early/2015/12/14/crl15-841.abstract

University-based open access publishing

This report, prepared for SPARC Europe, sketches the landscape of university-­‐based not-­‐for-­‐ profit publishing in Europe with a primary focus on open access publishing of journals. It provides a view of the different types of initiatives in terms of their size, operational and business models, technologies used, stakeholder involvement, concentration of scientific fields, growth, as well as regional characteristics and recommendations for SPARC Europe and DOAJ.

The report attests to a rich and continuously evolving ecology of open access publishing initiatives in universities in Europe and elsewhere. Beyond the commercial publishing models, it appears that university libraries are largely the foci of intense activity in journal publishing and books (primarily where a university press exists), while national governments are moving towards building national collections, national portals and services paid for by public funds to make research published within the country more relevant and accessible internationally.

This ecology is primarily populated by small publishers who are largely invisible, and much smaller numbers of large and medium-­‐ sized university-­‐based activities. At the same time, a growing number of innovative initiatives in the University and outside, mostly initiated by scholars and University Presses, eager to experiment in developing a fair and sustainable scholarly communications system, attests to a vibrant and swiftly-­‐evolving landscape.

URL : University-based open access publishing

Alternative location : http://sparceurope.org/wp-content/uploads/2015/12/SE_UPublishing_Report_0315.pdf