[Paper Review] A Complete Survey on LLM-based AI Chatbots

Sumit Kumar Dam, Choong Seon Hong+2 more|arXiv (Cornell University)|Jun 17, 2024

FinTech, Crowdfunding, Digital Finance18 citations

TL;DR

This survey comprehensively reviews the evolution, applications, challenges, and future directions of LLM-based chatbots across sectors, with a focus on data/tools, governance, and reliability.

ABSTRACT

The past few decades have witnessed an upsurge in data, forming the foundation for data-hungry, learning-based AI technology. Conversational agents, often referred to as AI chatbots, rely heavily on such data to train large language models (LLMs) and generate new content (knowledge) in response to user prompts. With the advent of OpenAI's ChatGPT, LLM-based chatbots have set new standards in the AI community. This paper presents a complete survey of the evolution and deployment of LLM-based chatbots in various sectors. We first summarize the development of foundational chatbots, followed by the evolution of LLMs, and then provide an overview of LLM-based chatbots currently in use and those in the development phase. Recognizing AI chatbots as tools for generating new knowledge, we explore their diverse applications across various industries. We then discuss the open challenges, considering how the data used to train the LLMs and the misuse of the generated knowledge can cause several issues. Finally, we explore the future outlook to augment their efficiency and reliability in numerous applications. By addressing key milestones and the present-day context of LLM-based chatbots, our survey invites readers to delve deeper into this realm, reflecting on how their next generation will reshape conversational AI.

Motivation & Objective

• Trace the historical evolution from pre-LLM chatbots to modern LLM-based chatbots
• Catalog and categorize current and in-development LLM-based chatbots beyond ChatGPT
• Analyze applications of LLM-based chatbots across education, research, healthcare, software engineering, and finance
• Identify technical and ethical challenges associated with data, knowledge generation, and deployment
• Discuss future directions to improve efficiency, reliability, and responsible use

Proposed method

• Review and synthesize foundational chatbot history and LLM architectures
• Present a taxonomy of LLM-based chatbots by functionality and sector
• Summarize technical concepts such as transformer architectures, in-context learning, and chain-of-thought prompting
• Analyze limitations and misuse concerns including data transparency, bias, privacy, and misinformation
• Provide future outlook recommendations for model optimization and ethical guidelines

Experimental results

Research questions

• RQ1 How have chatbots evolved from early systems to LLM-based variants and what foundational LLM advancements enabled this shift?
• RQ2 What are the key applications of LLM-based chatbots across sectors and how do they affect operations and user interactions?
• RQ3 What challenges affect the performance, reliability, and safe deployment of LLM-based chatbots?
• RQ4 What technical improvements and ethical guidelines are needed to ensure responsible use and governance of LLM-based chatbots?

Key findings

• LLMs and transformer-based architectures have driven a shift to richer, context-aware conversational capabilities and in-context learning for new tasks
• A broad taxonomy of chatbot applications is provided, spanning education, research, healthcare, software engineering, and finance
• The survey covers multiple chatbots (ChatGPT, BARD/Gemini, Bing Chat, Claude, Ernie Bot, BlenderBot, and others) beyond a single model
• Major challenges identified include knowledge recency, hallucinations, data transparency, bias, privacy, and misuse risks
• The paper highlights the potential benefits and risks of knowledge generation through chatbots and calls for ethical guidelines and responsible deployment