Archives des mots-clés : Artificial intelligence

Methodology for AI-Based Search Strategy of Scientific Papers: Exemplary Search for Hybrid and Battery Electric Vehicles in the Semantic Scholar Database

Posté le par

Authors : Florian Wätzold, Bartosz Popiela, Jonas Mayer

The rapid development of artificial intelligence (AI) has significantly enhanced productivity, particularly in repetitive tasks. In the scientific domain, literature review stands out as a key area where AI-based tools can be effectively applied. This study presents a methodology for developing a search strategy for systematic reviews using AI tools. The Semantic Scholar database served as the foundation for the search process. The methodology was tested by searching for scientific papers related to batteries and hydrogen vehicles with the aim of enabling an evaluation for their potential applications. An extensive list of vehicles and their operational environments based on international standards and literature reviews was defined and used as the main input for the exemplary search.

The AI-supported search yielded approximately 60,000 results, which were subjected to an initial relevance assessment. For the relevant papers, a neighbourhood analysis based on citation and reference networks was conducted. The final selection of papers, covering the period from 2013 to 2023, included 713 papers assessed after the initial review. An extensive discussion of the results is provided, including their categorisation based on search terms, publication years, and cluster analysis of powertrains, as well as operational environments of the vehicles involved.

This case study illustrates the effectiveness of the proposed methodology and serves as a starting point for future research. The results demonstrate the potential of AI-based tools to enhance productivity when searching for scientific papers.

URL : Methodology for AI-Based Search Strategy of Scientific Papers: Exemplary Search for Hybrid and Battery Electric Vehicles in the Semantic Scholar Database

DOI : https://doi.org/10.3390/publications12040049

Publication Trends in Artificial Intelligence Conferences: The Rise of Super Prolific Authors

Posté le par

Authors : Ariful Azad, Afeefa Banu

Papers published in top conferences contribute influential discoveries that are reshaping the landscape of modern Artificial Intelligence (AI). We analyzed 87,137 papers from 11 AI conferences to examine publication trends over the past decade. Our findings reveal a consistent increase in both the number of papers and authors, reflecting the growing interest in AI research.

We also observed a rise in prolific researchers who publish dozens of papers at the same conference each year. In light of this analysis, the AI research community should consider revisiting authorship policies, addressing equity concerns, and evaluating the workload of junior researchers to foster a more sustainable and inclusive research environment.

Arxiv : https://arxiv.org/abs/2412.07793

Evaluating Research Quality with Large Language Models: An Analysis of ChatGPT’s Effectiveness with Different Settings and Inputs

Posté le par

Author : Mike Thelwall

Evaluating the quality of academic journal articles is a time consuming but critical task for national research evaluation exercises, appointments and promotion. It is therefore important to investigate whether Large Language Models (LLMs) can play a role in this process.

This article assesses which ChatGPT inputs (full text without tables, figures and references; title and abstract; title only) produce better quality score estimates, and the extent to which scores are affected by ChatGPT models and system prompts.

The results show that the optimal input is the article title and abstract, with average ChatGPT scores based on these (30 iterations on a dataset of 51 papers) correlating at 0.67 with human scores, the highest ever reported. ChatGPT 4o is slightly better than 3.5-turbo (0.66), and 4o-mini (0.66).

The results suggest that article full texts might confuse LLM research quality evaluations, even though complex system instructions for the task are more effective than simple ones.

Thus, whilst abstracts contain insufficient information for a thorough assessment of rigour, they may contain strong pointers about originality and significance. Finally, linear regression can be used to convert the model scores into the human scale scores, which is 31% more accurate than guessing.

Arxiv : https://arxiv.org/abs/2408.06752

Enhancing peer review efficiency: A mixed-methods analysis of artificial intelligence-assisted reviewer selection across academic disciplines

Posté le par

Author : Shai Farber

This mixed-methods study evaluates the efficacy of artificial intelligence (AI)-assisted reviewer selection in academic publishing across diverse disciplines. Twenty journal editors assessed AI-generated reviewer recommendations for a manuscript. The AI system achieved a 42% overlap with editors’ selections and demonstrated a significant improvement in time efficiency, reducing selection time by 73%.

Editors found that 37% of AI-suggested reviewers who were not part of their initial selection were indeed suitable. The system’s performance varied across disciplines, with higher accuracy in STEM fields (Cohen’s d = 0.68). Qualitative feedback revealed an appreciation for the AI’s ability to identify lesser-known experts but concerns about its grasp of interdisciplinary work. Ethical considerations, including potential algorithmic bias and privacy issues, were highlighted.

The study concludes that while AI shows promise in enhancing reviewer selection efficiency and broadening the reviewer pool, it requires human oversight to address limitations in understanding nuanced disciplinary contexts. Future research should focus on larger-scale longitudinal studies and developing ethical frameworks for AI integration in peer-review processes.

URL : Enhancing peer review efficiency: A mixed-methods analysis of artificial intelligence-assisted reviewer selection across academic disciplines

DOI : https://doi.org/10.1002/leap.1638

The impact of generative AI on the scholarly communications of early career researchers: An international, multi-disciplinary study

Posté le par

Authors : David Nicholas, Marzena Swigon, David Clark, Abdullah Abrizah, Jorge Revez, Eti Herman, Blanca Rodríguez Bravo, Jie Xu, Anthony Watkinson

The Harbingers study of early career researchers (ECRs), their work life and scholarly communications, began by studying generational—Millennial—change (c.2016), then moved to pandemic change (c.2020) and is now investigating another potential agent of change: artificial intelligence (2024–). We report here on a substantial scoping pilot study that looks at the impact of AI on the scholarly communications of international ECRs and, extends this to the arts and humanities.

It aims to fill the knowledge gap concerning ECRs whose millennial mindset may render them especially open to change and, as the research workhorses they are, very much in the frontline. The data was collected via in-depth interviews in China, Malaysia, Poland, Portugal, Spain and (selectively) the United Kingdom/United States. The data show ECRs to be thinking, probing and, in some cases, experimenting with AI.

There was a general acceptance that AI will be responsible for the growth of low-quality scientific papers, which could lead to a decline in the quality of research. Scholarly integrity and ethics were a big concern with issues of authenticity, plagiarism, copyright and poor citation practices raised. The most widespread belief was AI would prove to be a transformative force and would exacerbate existing scholarly disparities and inequalities.

URL : The impact of generative AI on the scholarly communications of early career researchers: An international, multi-disciplinary study

DOI : https://doi.org/10.1002/leap.1628

How to make sense of generative AI as a science communication researcher? A conceptual framework in the context of critical engagement with scientific information

Posté le par

Authors : Inbal Klein-Avraham, Esther Greussing, Monika Taddicken, Shakked Dabran-Zivan, Evelyn Jonas, Ayelet Baram-Tsabari

A guiding theory for a continuous and cohesive discussion regarding generative artificial intelligence (GenAI) in science communication is still unavailable. Here, we propose a framework for characterizing, evaluating, and comparing AI-based information technologies in the context of critical engagement with scientific information in online environments.

Hierarchically constructed, the framework observes technological properties, user experience, content presentation, and the context in which the technology is being used. Understandable and applicable for non-experts in AI systems, the framework affords a holistic yet practical assessment of various AI-based information technologies, providing both a reflection aid and a conceptual baseline for scholarly references.

URL : How to make sense of generative AI as a science communication researcher? A conceptual framework in the context of critical engagement with scientific information

DOI : https://doi.org/10.22323/2.23060205

Open Science at the Generative AI Turn: An Exploratory Analysis of Challenges and Opportunities

Posté le par

Authors : Mohammad Hosseini, Serge P.J.M. Horbach, Kristi L. Holmes, Tony Ross-Hellauer

Technology influences Open Science (OS) practices, because conducting science in transparent, accessible, and participatory ways requires tools/platforms for collaborative research and sharing results. Due to this direct relationship, characteristics of employed technologies directly impact OS objectives. Generative Artificial Intelligence (GenAI) models are increasingly used by researchers for tasks such as text refining, code generation/editing, reviewing literature, data curation/analysis.

GenAI promises substantial efficiency gains but is currently fraught with limitations that could negatively impact core OS values such as fairness, transparency and integrity, and harm various social actors. In this paper, we explore possible positive and negative impacts of GenAI on OS.

We use the taxonomy within the UNESCO Recommendation on Open Science to systematically explore the intersection of GenAI and OS. We conclude that using GenAI could advance key OS objectives by further broadening meaningful access to knowledge, enabling efficient use of infrastructure, improving engagement of societal actors, and enhancing dialogue among knowledge systems.

However, due to GenAI limitations, it could also compromise the integrity, equity, reproducibility, and reliability of research, while also having potential implications for the political economy of research and its infrastructure. Hence, sufficient checks, validation and critical assessments are essential when incorporating GenAI into research workflows.

URL : Open Science at the Generative AI Turn: An Exploratory Analysis of Challenges and Opportunities

DOI : https://doi.org/10.31235/osf.io/zns7g