Archives des mots-clés : scientific communication

Essential work, invisible workers: The role of digital curation in COVID-19 Open Science

Posté le par

Authors : Irene V. Pasquetto, Amina A. Abdu, Natascha Chtena

In this paper, we examine the role digital curation practices and practitioners played in facilitating open science (OS) initiatives amid the COVID-19 pandemic. In Summer 2023, we conducted a content analysis of available information regarding 50 OS initiatives that emerged—or substantially shifted their focus—between 2020 and 2022 to address COVID-19 related challenges. Despite growing recognition of the value of digital curation for the organization, dissemination, and preservation of scientific knowledge, our study reveals that digital curatorial work often remains invisible in pandemic OS initiatives.

In particular, we find that, even among those initiatives that greatly invested in digital curation work, digital curation is seldom mentioned in mission statements, and little is known about the rationales behind curatorial choices and the individuals responsible for the implementation of curatorial strategies. Given the important yet persistent invisibility of digital curatorial work, we propose a shift in how we conceptualize digital curation from a practice that merely “adds value” to research outputs to a practice of knowledge production.

We conclude with reflections on how iSchools can lead in professionalizing the field and offer suggestions for initial steps in that direction.

URL : Essential work, invisible workers: The role of digital curation in COVID-19 Open Science

DOI : https://doi.org/10.1002/asi.24965

Open Science Alternatives to Scopus and the Web of Science: A Case Study in Regional Resilience

Posté le par

Authors : Irina D. Turgel, Olga A. Chernova

The recent years have seen increasing support for open science in academic circles. However, the large number of scientometric databases calls into question the comparability of the search and analysis tools they provide.

Using the subject area of regional resilience as an example, in this study, the aim was to analyze the capabilities of widely used databases to serve as alternatives to Scopus and Web of Science in solving research problems.

As alternatives, in the present article, the following open, free scientometric databases were considered: AMiner, Wizdom.ai, the Lens, Dimensions, and OpenAlex. Their capabilities were demonstrated for the subject area under study, and the obtained results were compared.

The study results showed that alternative databases provide essential data on trends in scientific development. It is noteworthy that they largely replicate the provided data, supplementing and expanding them by using different types of data sources. However, open databases do not guarantee a high quality of materials and exhibit a relatively low level of metadata.

Thus, it is premature to abandon the use of Scopus and Web of Science in research activities. Since scientometric databases were developed in different contexts, they are characterized by structural and functional heterogeneity, which complicates their comparison.

Therefore, a selective approach should be adopted for the choice of scientometric databases, taking into account financial and other constraints, as well as the specifics of research problems.

URL : Open Science Alternatives to Scopus and the Web of Science: A Case Study in Regional Resilience

DOI : https://doi.org/10.3390/publications12040043

Evaluating Research Quality with Large Language Models: An Analysis of ChatGPT’s Effectiveness with Different Settings and Inputs

Posté le par

Author : Mike Thelwall

Evaluating the quality of academic journal articles is a time consuming but critical task for national research evaluation exercises, appointments and promotion. It is therefore important to investigate whether Large Language Models (LLMs) can play a role in this process.

This article assesses which ChatGPT inputs (full text without tables, figures and references; title and abstract; title only) produce better quality score estimates, and the extent to which scores are affected by ChatGPT models and system prompts.

The results show that the optimal input is the article title and abstract, with average ChatGPT scores based on these (30 iterations on a dataset of 51 papers) correlating at 0.67 with human scores, the highest ever reported. ChatGPT 4o is slightly better than 3.5-turbo (0.66), and 4o-mini (0.66).

The results suggest that article full texts might confuse LLM research quality evaluations, even though complex system instructions for the task are more effective than simple ones.

Thus, whilst abstracts contain insufficient information for a thorough assessment of rigour, they may contain strong pointers about originality and significance. Finally, linear regression can be used to convert the model scores into the human scale scores, which is 31% more accurate than guessing.

Arxiv : https://arxiv.org/abs/2408.06752

Does Open Access Foster Interdisciplinary Citation? Decomposing Open Access Citation Advantage

Posté le par

Authors : Kai Nishikawa, Akiyoshi Murakami

The existence of an open access (OA) citation advantage, that is, whether OA increases citations, has been a topic of interest for many years. Although numerous previous studies have focused on whether OA increases citations, expectations for OA go beyond that. One such expectation is the promotion of knowledge transfer across various fields.

This study aimed to clarify whether OA, especially gold OA, increases interdisciplinary citations in various natural science fields. Specifically, we measured the effect of OA on interdisciplinary and within-discipline citation counts by decomposing an existing metric of the OA citation advantage.

The results revealed that OA increases both interdisciplinary and within-discipline citations in many fields and increases only interdisciplinary citations in chemistry, computer science, and clinical medicine. Among these fields, clinical medicine tends to obtain more interdisciplinary citations without being influenced by specific journals or papers.

The findings indicate that OA fosters knowledge transfer to different fields, which extends our understanding of its effects.

Arxiv : https://arxiv.org/abs/2411.14653

Open scholarship and bibliodiversity

Posté le par

Authors : Maureen P. Walsh, Nataliia Kaliuzhna, Nokuthula Mchunu, Mohamad Mostafa, Katherine Witzig, Tony Alves

This paper is based on the Open Scholarship and Bibliodiversity panel presented at the 2024 NISO Plus conference in Baltimore, Maryland on February 13, 2024, and brings together five perspectives on the interdependency of open scholarship and bibliodiversity. Bibliodiversity in the context of open scholarship refers to the diversity of publishing models, platforms, and formats that are available for scholarly communication.

It emphasizes the importance of a varied and inclusive ecosystem for acquiring academic knowledge and for the dissemination of research. An important part of bibliodiversity is the inclusion and the promotion of a diversity of scholarly voices.

The authors explore how to ensure that a scholarly infrastructure includes a multitude of voices, is accessible to everyone, and can be expressed in a variety of ways.

URL : Open scholarship and bibliodiversity

DOI : https://doi.org/10.1177/18758789241296760

Fundamental problems in the peer-review process and stakeholders’ perceptions of potential suggestions for improvement

Posté le par

Authors : Cigdem Kadaifci, Erkan Isikli, Y. Ilker Topcu

Academic papers are essential for researchers to communicate their work to their peers and industry experts. Quality research is published in prestigious scientific journals, and is considered as part of the hiring and promotion criteria at leading universities. Scientific journals conduct impartial and anonymous peer reviews of submitted manuscripts; however, individuals involved in this process may encounter issues related to the duration, impartiality, and transparency of these reviews.

To explore these concerns, we created a questionnaire based on a comprehensive review of related literature and expert opinions, which was distributed to all stakeholders (authors, reviewers, and editors) who participated in the peer-review process from a variety of countries and disciplines. Their opinions on the primary issues during the process and suggestions for improvement were collected. The data were then analysed based on various groups, such as gender, country of residence, and contribution type, using appropriate multivariate statistical techniques to determine the perceptions and experiences of participants in the peer-review process.

The results showed that unethical behaviour was not uncommon and that editors and experienced reviewers encountered it more frequently. Women and academics from Türkiye were more likely to experience ethical violations and perceived them as more ethically severe. Incentives and stakeholder involvement were seen as ways to enhance the quality and impartiality of peer review. The scale developed can serve as a useful tool for addressing difficulties in the peer-review process and improving its effectiveness and performance.

URL : Fundamental problems in the peer-review process and stakeholders’ perceptions of potential suggestions for improvement

DOI : https://doi.org/10.1002/leap.1637

Adoption and use of author identifier services: A French national survey

Posté le par

Authors : Christophe Boudry, Aline Bouchard

This paper studies awareness and use of author identifier services (AIDs) in the French academic community and explores needs and forms of support required for these tools, using a national questionnaire survey. ArXivID, IdHAL, ORCID, ResearcherID and Scopus Author ID were investigated. A total of 6125 people completed the questionnaire in full. The results of this survey show that discipline and age play an important role in French researchers’ familiarity with AIDs.

IdHAL and ORCID were by far the two best known AIDs, probably because they have been promoted by institutions in France for several years. French researchers use AIDs mainly to respond to external requests (e.g., to submit an article or a research project), while, surprisingly, few use them to ‘facilitate their work’.

When French researchers were asked about their needs and the form of support required for AIDs, more than 30% of them said they either required an introduction to or practical training in these tools. The results of this national survey should help stakeholders to adapt their policies and to guide and support researchers more efficiently in the use of these tools.

URL : Adoption and use of author identifier services: A French national survey

DOI : https://doi.org/10.1002/leap.1640