[Paper Review] Chatbots and ChatGPT: A Bibliometric Analysis and Systematic Review of Publications in Web of Science and Scopus Databases

Hamed Khosravi, Mohammad Reza Shafie+3 more|arXiv (Cornell University)|Apr 11, 2023

Artificial Intelligence in Healthcare and Education14 citations

TL;DR

The paper conducts a two-phase bibliometric study of chatbot literature and ChatGPT publications in Scopus and WoS (1998–2023), identifying growth trends, key topics, and future research directions.

ABSTRACT

This paper presents a bibliometric analysis of the scientific literature related to chatbots, focusing specifically on ChatGPT. Chatbots have gained increasing attention recently, with an annual growth rate of 19.16% and 27.19% on the Web of Sciences (WoS) and Scopus, respectively. In this study, we have explored the structure, conceptual evolution, and trends in this field by analyzing data from both Scopus and WoS databases. The research consists of two study phases: (i) an analysis of chatbot literature and (ii) a comprehensive review of scientific documents on ChatGPT. In the first phase, a bibliometric analysis is conducted on all published literature, including articles, book chapters, conference papers, and reviews on chatbots from both Scopus (5839) and WoS (2531) databases covering the period from 1998 to 2023. An in-depth analysis focusing on sources, countries, authors' impact, and keywords has revealed that ChatGPT is the latest trend in the chatbot field. Consequently, in the second phase, bibliometric analysis has been carried out on ChatGPT publications, and 45 published studies have been analyzed thoroughly based on their methods, novelty, and conclusions. The key areas of interest identified from the study can be classified into three groups: artificial intelligence and related technologies, design and evaluation of conversational agents, and digital technologies and mental health. Overall, the study aims to provide guidelines for researchers to conduct their research more effectively in the field of chatbots and specifically highlight significant areas for future investigation into ChatGPT.

Motivation & Objective

• Assess the structure and evolution of chatbot-related literature across Scopus and Web of Science databases.
• Identify growth rates, prolific sources, countries, authors, and keywords in chatbot research from 1998 to 2023.
• Characterize the emergence and trajectory of ChatGPT-focused publications and synthesize insights to guide future research.

Proposed method

• Two-phase study design with (i) broad bibliometric analysis of all chatbot literature and (ii) in-depth review of ChatGPT publications.
• Data extraction from Scopus (5839 items) and WoS (2531 items) covering 1998–2023.
• Analysis of sources, countries, authors’ impact, and keywords to identify trends and hotspots.
• Focused bibliometric analysis of 45 ChatGPT publications to evaluate methods, novelty, and conclusions.

Experimental results

Research questions

• RQ1 What is the growth, structure, and evolution of chatbot-related publications in Scopus and WoS from 1998 to 2023?
• RQ2 What are the main sources, countries, authors, and keywords shaping chatbot research trajectories?
• RQ3 What is the status and trajectory of ChatGPT-specific publications, and what methodologies and conclusions characterize them?
• RQ4 What key areas and gaps emerge at the intersection of AI, conversational agent design, and digital technologies and mental health?

Key findings

• Chatbots show an annual growth rate of 19.16% in Web of Science and 27.19% in Scopus.
• ChatGPT is identified as the latest prominent trend within the chatbot field.
• Phase two analyzes 45 ChatGPT publications in depth to assess methods, novelty, and conclusions.
• Three major interest areas emerge: artificial intelligence and related technologies; design and evaluation of conversational agents; and digital technologies and mental health.
• The study provides guidelines to researchers and highlights significant future investigation areas for ChatGPT.