Academic writing in the age of AI: Comparing the reliability of ChatGPT and Bard with Scopus and Web of Science

Authors : Swati Garg, Asad Ahmad, Dag Øivind Madsen

ChatGPT and Bard (now known as Gemini) are becoming indispensable resources for researchers, academicians and diverse stakeholders within the academic landscape. At the same time, traditional digital tools such as scholarly databases continue to be widely used. Web of Science and Scopus are the most extensive academic databases and are generally regarded as consistently reliable scholarly research resources. With the increasing acceptance of artificial intelligence (AI) in academic writing, this study focuses on understanding the reliability of the new AI models compared to Scopus and Web of Science.

The study includes a bibliometric analysis of green, sustainable and ecological buying behaviour, covering the period from 1 January 2011 to 21 May 2023. These results are used to compare the results from the AI and the traditional scholarly databases on several parameters. Overall, the findings suggest that AI models like ChatGPT and Bard are not yet reliable for academic writing tasks. It appears to be too early to depend on AI for such tasks.

URL : Academic writing in the age of AI: Comparing the reliability of ChatGPT and Bard with Scopus and Web of Science

DOI : https://doi.org/10.1016/j.jik.2024.100563

Distinguishing articles in questionable and non-questionable journals using quantitative indicators associated with quality

Author : Dimity Stephen

This study investigates the viability of distinguishing articles in questionable journals (QJs) from those in non-QJs on the basis of quantitative indicators typically associated with quality. Subsequently, I examine what can be deduced about the quality of articles in QJs based on the differences observed.

I contrast the length of abstracts and full-texts, prevalence of spelling errors, text readability, number of references and citations, the size and internationality of the author team, the documentation of ethics and informed consent statements, and the presence erroneous decisions based on statistical errors in 1,714 articles from 31 QJs, 1,691 articles from 16 journals indexed in Web of Science (WoS), and 1,900 articles from 45 mid-tier journals, all in the field of psychology.

The results suggest that QJ articles do diverge from the disciplinary standards set by peer-reviewed journals in psychology on quantitative indicators of quality that tend to reflect the effect of peer review and editorial processes. However, mid-tier and WoS journals are also affected by potential quality concerns, such as under-reporting of ethics and informed consent processes and the presence of errors in interpreting statistics. Further research is required to develop a comprehensive understanding of the quality of articles in QJs.

Arxiv : https://arxiv.org/abs/2405.06308

From indexation policies through citation networks to normalized citation impacts: Web of Science, Scopus, and Dimensions as varying resonance chambers

Authors : Stephan Stahlschmidt, Dimity Stephen

Dimensions was introduced as an alternative bibliometric database to the well-established Web of Science (WoS) and Scopus, however all three databases have fundamental differences in coverage and content, resultant from their owners’ indexation philosophies.

In light of these differences, we explore here, using a citation network analysis and assessment of normalised citation impact of “duplicate” publications, whether the three databases offer structurally different perspectives of the bibliometric landscape or if they are essentially homogenous substitutes.

Our citation network analysis of core and exclusive 2016-2018 publications revealed a large set of core publications indexed in all three databases that are highly self-referential. In comparison, each database selected a set of exclusive publications that appeared to hold similarly low levels of relevance to the core set and to one another, with slightly more internal communication between exclusive publications in Scopus and Dimensions than WoS.

Our comparison of normalised citations for 41,848 publications indexed in all three databases found that German sectors were valuated as more impactful in Scopus and Dimensions compared to WoS, particularly for sectors with an applied research focus.

We conclude that the databases do present structurally different perspectives, although Scopus and Dimensions with their additional circle of applied research vary more from the more base research-focused WoS than they do from one another.

URL : https://arxiv.org/abs/2106.01695

Large-scale comparison of bibliographic data sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic

Authors : Martijn Visser, Nees Jan van Eck, Ludo Waltman

We present a large-scale comparison of five multidisciplinary bibliographic data sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. The comparison considers scientific documents from the period 2008-2017 covered by these data sources. Scopus is compared in a pairwise manner with each of the other data sources.

We first analyze differences between the data sources in the coverage of documents, focusing for instance on differences over time, differences per document type, and differences per discipline.

We then study differences in the completeness and accuracy of citation links. Based on our analysis, we discuss strengths and weaknesses of the different data sources. We emphasize the importance of combining a comprehensive coverage of the scientific literature with a flexible set of filters for making selections of the literature.

URL : https://arxiv.org/abs/2005.10732

Open Bioeconomy—A Bibliometric Study on the Accessibility of Articles in the Field of Bioeconomy

Authors : Marianne Duquenne, Hélène Pros, Joachim Schöpfel, Franck Dumeignil

Open access (OA) to scientific information is one of the major challenges and objectives of actual public research policy. The purpose of this paper is to assess the degree of openness of scientific articles on bioeconomy, as one of the emergent research fields at the crossroads of several disciplines and with high societal and industrial impact.

Based on a Web of Science (WoS) corpus of 2489 articles published between 2015 and 2019, we calculated bibliometric indicators, explored the openness of each article and assessed the share of journals, countries and research areas of these articles.

The results show a sharp increase and diversification of articles in the field of bioeconomy, with a beginning long tail distribution. 45.6% of the articles are freely available and the share of OA articles is steadily increasing, from 31% in 2015 to 52% in 2019.

Gold is the most important variant of OA. Open access is low in the applied research areas of chemical, agricultural and environmental engineering but higher in the domains of energy and fuels, forestry and green and sustainable science and technology.

The UK and the Netherlands have the highest rates of OA articles, followed by Spain and Germany. The funding rate of OA articles is higher than of non-OA articles. This is the first bibliometric study on open access to articles on bioeconomy.

The results can be useful for the further development of OA editorial and funding criteria in the field of bioeconomy.

URL : Open Bioeconomy—A Bibliometric Study on the Accessibility of Articles in the Field of Bioeconomy

DOI : https://doi.org/10.3390/publications8040055

Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations

Authors : Alberto Martín-Martín, Mike Thelwall, Enrique Orduna-Malea, Emilio Delgado López-Cózar

New sources of citation data have recently become available, such as Microsoft Academic, Dimensions, and the OpenCitations Index of CrossRef open DOI-to-DOI citations (COCI). Although these have been compared to the Web of Science Core Collection (WoS), Scopus, or Google Scholar, there is no systematic evidence of their differences across subject categories.

In response, this paper investigates 3,073,351 citations found by these six data sources to 2,515 English-language highly-cited documents published in 2006 from 252 subject categories, expanding and updating the largest previous study. Google Scholar found 88% of all citations, many of which were not found by the other sources, and nearly all citations found by the remaining sources (89–94%).

A similar pattern held within most subject categories. Microsoft Academic is the second largest overall (60% of all citations), including 82% of Scopus citations and 86% of WoS citations. In most categories, Microsoft Academic found more citations than Scopus and WoS (182 and 223 subject categories, respectively), but had coverage gaps in some areas, such as Physics and some Humanities categories. After Scopus, Dimensions is fourth largest (54% of all citations), including 84% of Scopus citations and 88% of WoS citations.

It found more citations than Scopus in 36 categories, more than WoS in 185, and displays some coverage gaps, especially in the Humanities. Following WoS, COCI is the smallest, with 28% of all citations. Google Scholar is still the most comprehensive source. In many subject categories Microsoft Academic and Dimensions are good alternatives to Scopus and WoS in terms of coverage.

DOI : https://doi.org/10.1007/s11192-020-03690-4

Competence-Based Management Research in the Web of Science and Scopus Databases: Scientific Production, Collaboration, and Impact

Authors : Vítor Vasata Macchi Silva, José Luis Duarte Ribeiro, Gonzalo Rubén Alvarez, Sonia Elisa Caregnato

This paper presents a bibliometric study, which seeks to characterize papers that address competence-based management and that are indexed in the Web of Science and Scopus databases in terms of scientific production, collaboration, and impact.

All the papers published in journals or in conference proceedings that contained the terms “competenc* management” or “compentenc* based management” in their titles, abstracts, or keywords were analyzed.

The results show that computational sciences, human resources management, strategic management, and industrial relations and labor correspond to the macro-categories that characterize competence-based management.

This paper also indicates that collaborations between authors do not establish strong co-authorship networks. It also shows that the most cited papers were published in journals of different areas.

It concludes that studies conducted in the area of competence-based management can be developed in a more assertive way if they take into consideration the context of the current state of research in this area.

URL : Competence-Based Management Research in the Web of Science and Scopus Databases: Scientific Production, Collaboration, and Impact

Alternative location : https://www.mdpi.com/2304-6775/7/4/60