Étiquette : Artificial intelligence

Catégories
EN

AI and Open Science: Implications and Library Practice

Author : Nicole Helregel

With the increasing proliferation of artificial intelligence (AI) in higher education and science, technology, engineering, and mathematics research, what are the implications for open science?

As the open science movement advocates for increased transparency and openness in the research process, where do AI and machine learning fit in? And where does that leave library and information science professionals in roles related to open science?

This article explores several approaches and considerations for how AI impacts open science, including whether AI has sufficient openness and transparency to align with the goals of open science, whether AI can be used to further open science goals, and the effects of AI use on researcher and public attitudes and actions.

The article provides recommendations for library practice, including knowledge-building, connections and advocacy, consultations and liaison work, licensing, and science communication and engagement.

URL : AI and Open Science: Implications and Library Practice

DOI : https://dx.doi.org/10.1353/lib.2025.a961191

Catégories
EN

Assessing the Societal Impact of Academic Research With Artificial Intelligence (AI): A Scoping Review of Business School Scholarship as a ‘Force for Good’

Authors : David Steingard, Kathleen Rodenburg

This study addresses critical questions about how current evaluative frameworks for academic research can effectively translate scholarly findings into practical applications and policies to tackle societal ‘grand challenges’.

This scoping review analysis was conducted using bibliometric methods and AI tools. Articles were drawn from a wide range of disciplines, with particular emphasis on the business and management fields, focusing on the burgeoning scholarship area of ‘business as a force for good’.

The novel integration of generative AI research approaches underscores the transformative potential of AI-human collaboration in academic research. Metadata from 4051 articles were examined in the scoping review, with only 370 articles (9.1%) explicitly identified as relevant to societal impact.

This finding reveals a substantial and concerning gap in research addressing the urgent social and environmental issues of our time. To address this gap, the study identifies six meta-themes related to enhancing the societal impact of research: business applications; faculty publication pressure; societal impact focus; sustainable development; university and scholarly rankings; and reference to responsible research frameworks.

Key findings highlight critical misalignments between research outputs and the United Nations Sustainable Development Goals (SDGs) and a lack of practical business applications of research insights.

The results emphasise the urgent need for academic institutions to expand evaluation criteria beyond traditional metrics to prioritise real-world impacts. Recommendations include developing holistic evaluation frameworks and incentivising research that addresses pressing societal challenges—shifting academia from a ‘scholar-to-scholar’ to a ‘scholar-to-society’ paradigm.

The implications of this shift are applied to business-related scholarship and its potential to inspire meaningful societal impact through business practice.

URL : Assessing the Societal Impact of Academic Research With Artificial Intelligence (AI): A Scoping Review of Business School Scholarship as a ‘Force for Good’

DOI : https://doi.org/10.1002/leap.2010

Catégories
EN

Evaluating the predictive capacity of ChatGPT for academic peer review outcomes across multiple platforms

Authors : Mike Thelwall, Abdallah Yaghi

Academic peer review is at the heart of scientific quality control, yet the process is slow and time-consuming. Technology that can predict peer review outcomes may help with this, for example by fast-tracking desk rejection decisions. While previous studies have demonstrated that Large Language Models (LLMs) can predict peer review outcomes to some extent, this paper introduces two new contexts and employs a more robust method—averaging multiple ChatGPT scores.

Averaging 30 ChatGPT predictions, based on reviewer guidelines and using only the submitted titles and abstracts failed to predict peer review outcomes for F1000Research (Spearman’s rho = 0.00). However, it produced mostly weak positive correlations with the quality dimensions of SciPost Physics (rho = 0.25 for validity, rho = 0.25 for originality, rho = 0.20 for significance, and rho = 0.08 for clarity) and a moderate positive correlation for papers from the International Conference on Learning Representations (ICLR) (rho = 0.38). Including article full texts increased the correlation for ICLR (rho = 0.46) and slightly improved it for F1000Research (rho = 0.09), with variable effects on the four quality dimension correlations for SciPost LaTeX files.

The use of simple chain-of-thought system prompts slightly increased the correlation for F1000Research (rho = 0.10), marginally reduced it for ICLR (rho = 0.37), and further decreased it for SciPost Physics (rho = 0.16 for validity, rho = 0.18 for originality, rho = 0.18 for significance, and rho = 0.05 for clarity). Overall, the results suggest that in some contexts, ChatGPT can produce weak pre-publication quality predictions.

However, their effectiveness and the optimal strategies for employing them vary considerably between platforms, journals, and conferences. Finally, the most suitable inputs for ChatGPT appear to differ depending on the platform.

URL : Evaluating the predictive capacity of ChatGPT for academic peer review outcomes across multiple platforms

DOI : https://doi.org/10.1007/s11192-025-05287-1

Catégories
EN

Use of artificial intelligence innovations in public academic libraries

Authors : Amogelang Isaac Molaudzi, Patrick Ngulube

Public academic libraries are among the many organisations concerned about using artificial intelligence (AI) technologies. The study adopted a mixed methods research (MMR) approach using a concurrent research design to examine the use of AI innovations in public academic libraries. Thematic and descriptive statistical data analysis was used to analyse the data gathered from questionnaires, interviews and document content analysis. The findings revealed that public academic libraries in South Africa did not have clear strategies for adopting AI innovations.

Consequently, AI was not widely used. Library management systems can support AI, but some must be upgraded. Librarians had excellent computer literacy, although many had not received AI training to broaden their expertise and awareness of this innovation. Results suggested that public academic libraries should create comprehensive AI adoption strategies responsive to AI trends.

This study highlights the need for strategies that ensure AI technologies are utilized ethically, equitably, and with accountability. It also contributes to the literature on the use of AI in academic libraries. The results of this study may encourage public academic librarians to begin planning the incorporation of AI technology into their strategies.

URL : Use of artificial intelligence innovations in public academic libraries

DOI : https://doi.org/10.1177/0340035224130178

Catégories
EN

Methodology for AI-Based Search Strategy of Scientific Papers: Exemplary Search for Hybrid and Battery Electric Vehicles in the Semantic Scholar Database

Authors : Florian Wätzold, Bartosz Popiela, Jonas Mayer

The rapid development of artificial intelligence (AI) has significantly enhanced productivity, particularly in repetitive tasks. In the scientific domain, literature review stands out as a key area where AI-based tools can be effectively applied. This study presents a methodology for developing a search strategy for systematic reviews using AI tools. The Semantic Scholar database served as the foundation for the search process. The methodology was tested by searching for scientific papers related to batteries and hydrogen vehicles with the aim of enabling an evaluation for their potential applications. An extensive list of vehicles and their operational environments based on international standards and literature reviews was defined and used as the main input for the exemplary search.

The AI-supported search yielded approximately 60,000 results, which were subjected to an initial relevance assessment. For the relevant papers, a neighbourhood analysis based on citation and reference networks was conducted. The final selection of papers, covering the period from 2013 to 2023, included 713 papers assessed after the initial review. An extensive discussion of the results is provided, including their categorisation based on search terms, publication years, and cluster analysis of powertrains, as well as operational environments of the vehicles involved.

This case study illustrates the effectiveness of the proposed methodology and serves as a starting point for future research. The results demonstrate the potential of AI-based tools to enhance productivity when searching for scientific papers.

URL : Methodology for AI-Based Search Strategy of Scientific Papers: Exemplary Search for Hybrid and Battery Electric Vehicles in the Semantic Scholar Database

DOI : https://doi.org/10.3390/publications12040049

Catégories
EN

Publication Trends in Artificial Intelligence Conferences: The Rise of Super Prolific Authors

Authors : Ariful Azad, Afeefa Banu

Papers published in top conferences contribute influential discoveries that are reshaping the landscape of modern Artificial Intelligence (AI). We analyzed 87,137 papers from 11 AI conferences to examine publication trends over the past decade. Our findings reveal a consistent increase in both the number of papers and authors, reflecting the growing interest in AI research.

We also observed a rise in prolific researchers who publish dozens of papers at the same conference each year. In light of this analysis, the AI research community should consider revisiting authorship policies, addressing equity concerns, and evaluating the workload of junior researchers to foster a more sustainable and inclusive research environment.

Arxiv : https://arxiv.org/abs/2412.07793

Catégories
EN

Evaluating Research Quality with Large Language Models: An Analysis of ChatGPT’s Effectiveness with Different Settings and Inputs

Author : Mike Thelwall

Evaluating the quality of academic journal articles is a time consuming but critical task for national research evaluation exercises, appointments and promotion. It is therefore important to investigate whether Large Language Models (LLMs) can play a role in this process.

This article assesses which ChatGPT inputs (full text without tables, figures and references; title and abstract; title only) produce better quality score estimates, and the extent to which scores are affected by ChatGPT models and system prompts.

The results show that the optimal input is the article title and abstract, with average ChatGPT scores based on these (30 iterations on a dataset of 51 papers) correlating at 0.67 with human scores, the highest ever reported. ChatGPT 4o is slightly better than 3.5-turbo (0.66), and 4o-mini (0.66).

The results suggest that article full texts might confuse LLM research quality evaluations, even though complex system instructions for the task are more effective than simple ones.

Thus, whilst abstracts contain insufficient information for a thorough assessment of rigour, they may contain strong pointers about originality and significance. Finally, linear regression can be used to convert the model scores into the human scale scores, which is 31% more accurate than guessing.

Arxiv : https://arxiv.org/abs/2408.06752