Archives des mots-clés : scientific communication

Open Science Alternatives to Scopus and the Web of Science: A Case Study in Regional Resilience

Posté le par

Authors : Irina D. Turgel, Olga A. Chernova

The recent years have seen increasing support for open science in academic circles. However, the large number of scientometric databases calls into question the comparability of the search and analysis tools they provide.

Using the subject area of regional resilience as an example, in this study, the aim was to analyze the capabilities of widely used databases to serve as alternatives to Scopus and Web of Science in solving research problems.

As alternatives, in the present article, the following open, free scientometric databases were considered: AMiner, Wizdom.ai, the Lens, Dimensions, and OpenAlex. Their capabilities were demonstrated for the subject area under study, and the obtained results were compared.

The study results showed that alternative databases provide essential data on trends in scientific development. It is noteworthy that they largely replicate the provided data, supplementing and expanding them by using different types of data sources. However, open databases do not guarantee a high quality of materials and exhibit a relatively low level of metadata.

Thus, it is premature to abandon the use of Scopus and Web of Science in research activities. Since scientometric databases were developed in different contexts, they are characterized by structural and functional heterogeneity, which complicates their comparison.

Therefore, a selective approach should be adopted for the choice of scientometric databases, taking into account financial and other constraints, as well as the specifics of research problems.

URL : Open Science Alternatives to Scopus and the Web of Science: A Case Study in Regional Resilience

DOI : https://doi.org/10.3390/publications12040043

Evaluating Research Quality with Large Language Models: An Analysis of ChatGPT’s Effectiveness with Different Settings and Inputs

Posté le par

Author : Mike Thelwall

Evaluating the quality of academic journal articles is a time consuming but critical task for national research evaluation exercises, appointments and promotion. It is therefore important to investigate whether Large Language Models (LLMs) can play a role in this process.

This article assesses which ChatGPT inputs (full text without tables, figures and references; title and abstract; title only) produce better quality score estimates, and the extent to which scores are affected by ChatGPT models and system prompts.

The results show that the optimal input is the article title and abstract, with average ChatGPT scores based on these (30 iterations on a dataset of 51 papers) correlating at 0.67 with human scores, the highest ever reported. ChatGPT 4o is slightly better than 3.5-turbo (0.66), and 4o-mini (0.66).

The results suggest that article full texts might confuse LLM research quality evaluations, even though complex system instructions for the task are more effective than simple ones.

Thus, whilst abstracts contain insufficient information for a thorough assessment of rigour, they may contain strong pointers about originality and significance. Finally, linear regression can be used to convert the model scores into the human scale scores, which is 31% more accurate than guessing.

Arxiv : https://arxiv.org/abs/2408.06752

Does Open Access Foster Interdisciplinary Citation? Decomposing Open Access Citation Advantage

Posté le par

Authors : Kai Nishikawa, Akiyoshi Murakami

The existence of an open access (OA) citation advantage, that is, whether OA increases citations, has been a topic of interest for many years. Although numerous previous studies have focused on whether OA increases citations, expectations for OA go beyond that. One such expectation is the promotion of knowledge transfer across various fields.

This study aimed to clarify whether OA, especially gold OA, increases interdisciplinary citations in various natural science fields. Specifically, we measured the effect of OA on interdisciplinary and within-discipline citation counts by decomposing an existing metric of the OA citation advantage.

The results revealed that OA increases both interdisciplinary and within-discipline citations in many fields and increases only interdisciplinary citations in chemistry, computer science, and clinical medicine. Among these fields, clinical medicine tends to obtain more interdisciplinary citations without being influenced by specific journals or papers.

The findings indicate that OA fosters knowledge transfer to different fields, which extends our understanding of its effects.

Arxiv : https://arxiv.org/abs/2411.14653

Open scholarship and bibliodiversity

Posté le par

Authors : Maureen P. Walsh, Nataliia Kaliuzhna, Nokuthula Mchunu, Mohamad Mostafa, Katherine Witzig, Tony Alves

This paper is based on the Open Scholarship and Bibliodiversity panel presented at the 2024 NISO Plus conference in Baltimore, Maryland on February 13, 2024, and brings together five perspectives on the interdependency of open scholarship and bibliodiversity. Bibliodiversity in the context of open scholarship refers to the diversity of publishing models, platforms, and formats that are available for scholarly communication.

It emphasizes the importance of a varied and inclusive ecosystem for acquiring academic knowledge and for the dissemination of research. An important part of bibliodiversity is the inclusion and the promotion of a diversity of scholarly voices.

The authors explore how to ensure that a scholarly infrastructure includes a multitude of voices, is accessible to everyone, and can be expressed in a variety of ways.

URL : Open scholarship and bibliodiversity

DOI : https://doi.org/10.1177/18758789241296760

Fundamental problems in the peer-review process and stakeholders’ perceptions of potential suggestions for improvement

Posté le par

Authors : Cigdem Kadaifci, Erkan Isikli, Y. Ilker Topcu

Academic papers are essential for researchers to communicate their work to their peers and industry experts. Quality research is published in prestigious scientific journals, and is considered as part of the hiring and promotion criteria at leading universities. Scientific journals conduct impartial and anonymous peer reviews of submitted manuscripts; however, individuals involved in this process may encounter issues related to the duration, impartiality, and transparency of these reviews.

To explore these concerns, we created a questionnaire based on a comprehensive review of related literature and expert opinions, which was distributed to all stakeholders (authors, reviewers, and editors) who participated in the peer-review process from a variety of countries and disciplines. Their opinions on the primary issues during the process and suggestions for improvement were collected. The data were then analysed based on various groups, such as gender, country of residence, and contribution type, using appropriate multivariate statistical techniques to determine the perceptions and experiences of participants in the peer-review process.

The results showed that unethical behaviour was not uncommon and that editors and experienced reviewers encountered it more frequently. Women and academics from Türkiye were more likely to experience ethical violations and perceived them as more ethically severe. Incentives and stakeholder involvement were seen as ways to enhance the quality and impartiality of peer review. The scale developed can serve as a useful tool for addressing difficulties in the peer-review process and improving its effectiveness and performance.

URL : Fundamental problems in the peer-review process and stakeholders’ perceptions of potential suggestions for improvement

DOI : https://doi.org/10.1002/leap.1637

Adoption and use of author identifier services: A French national survey

Posté le par

Authors : Christophe Boudry, Aline Bouchard

This paper studies awareness and use of author identifier services (AIDs) in the French academic community and explores needs and forms of support required for these tools, using a national questionnaire survey. ArXivID, IdHAL, ORCID, ResearcherID and Scopus Author ID were investigated. A total of 6125 people completed the questionnaire in full. The results of this survey show that discipline and age play an important role in French researchers’ familiarity with AIDs.

IdHAL and ORCID were by far the two best known AIDs, probably because they have been promoted by institutions in France for several years. French researchers use AIDs mainly to respond to external requests (e.g., to submit an article or a research project), while, surprisingly, few use them to ‘facilitate their work’.

When French researchers were asked about their needs and the form of support required for AIDs, more than 30% of them said they either required an introduction to or practical training in these tools. The results of this national survey should help stakeholders to adapt their policies and to guide and support researchers more efficiently in the use of these tools.

URL : Adoption and use of author identifier services: A French national survey

DOI : https://doi.org/10.1002/leap.1640

Two scholarly publishing cultures? Open access drives a divergence in European academic publishing practices

Posté le par

Authors : Leon Kopitar, Nejc Plohl, Mojca Tancer Verboten, Gregor Štiglic, Roger Watson, Dean Korošak

The current system of scholarly publishing is often criticized for being slow, expensive, and not transparent. The rise of open access publishing as part of open science tenets, promoting transparency and collaboration, together with calls for research assesment reforms are the results of these criticisms. The emergence of new open access publishers presents a unique opportunity to empirically test how universities and countries respond to shifts in the academic publishing landscape. These new actors challenge traditional publishing models, offering faster review times and broader accessibility, which could influence strategic publishing decisions.

Our findings reveal a clear division in European publishing practices, with countries clustering into two groups distinguished by the ratio of publications in new open access journals with accelerated review times versus legacy journals. This divide underscores a broader shift in academic culture, highlighting new open access publishing venues as a strategic factor influencing national and institutional publishing practices, with significant implications for research accessibility and collaboration across Europe.

Arxiv : https://arxiv.org/abs/2411.06282