[Paper Review] Chatbots to ChatGPT in a Cybersecurity Space: Evolution, Vulnerabilities, Attacks, Challenges, and Future Recommendations

Attia Qammar, Hongmei Wang+4 more|arXiv (Cornell University)|May 29, 2023

Advanced Malware Detection Techniques20 citations

TL;DR

The paper surveys the evolution of chatbots to ChatGPT, analyzes cybersecurity vulnerabilities and attacks on chatbots (including ChatGPT), and discusses defenses and future directions.

ABSTRACT

Chatbots shifted from rule-based to artificial intelligence techniques and gained traction in medicine, shopping, customer services, food delivery, education, and research. OpenAI developed ChatGPT blizzard on the Internet as it crossed one million users within five days of its launch. However, with the enhanced popularity, chatbots experienced cybersecurity threats and vulnerabilities. This paper discussed the relevant literature, reports, and explanatory incident attacks generated against chatbots. Our initial point is to explore the timeline of chatbots from ELIZA (an early natural language processing computer program) to GPT-4 and provide the working mechanism of ChatGPT. Subsequently, we explored the cybersecurity attacks and vulnerabilities in chatbots. Besides, we investigated the ChatGPT, specifically in the context of creating the malware code, phishing emails, undetectable zero-day attacks, and generation of macros and LOLBINs. Furthermore, the history of cyberattacks and vulnerabilities exploited by cybercriminals are discussed, particularly considering the risk and vulnerabilities in ChatGPT. Addressing these threats and vulnerabilities requires specific strategies and measures to reduce the harmful consequences. Therefore, the future directions to address the challenges were presented.

Motivation & Objective

• Trace the historical development of chatbots from ELIZA to GPT-4 and explain ChatGPT’s working mechanism.
• Identify cybersecurity threats and vulnerabilities across chatbot modules (client, network, response generation, database).
• Examine ChatGPT as a case study for offensive cyber activities such as malware code generation, phishing, zero-day attacks, and LOLBINs.
• Summarize the history of cyberattacks against chatbots and discuss challenges and future directions for mitigation.

Proposed method

• Review and synthesize existing literature, incident reports, and case examples related to chatbot cybersecurity.
• Describe the working mechanisms of ChatGPT and the GPT series including SFT, reward modeling, and PPO.
• Categorize attacks and vulnerabilities by chatbot modules and provide proposed countermeasures.
• Present case studies of ChatGPT used to generate malicious code, phishing emails, and LOLBINs.

Experimental results

Research questions

• RQ1 What is the evolution trajectory of chatbots from ELIZA to GPT-4 and how does ChatGPT operate within this lineage?
• RQ2 What cybersecurity threats and vulnerabilities affect chatbot architectures, and how have attackers exploited ChatGPT specifically?
• RQ3 What defensive strategies and future directions can mitigate chatbot-related cyber threats, including offensive use cases of ChatGPT?
• RQ4 How do historical chatbot cyberattacks inform current risk assessments and mitigation practices?

Key findings

• Chatbots evolved from rule-based systems to AI-driven models (generative and retrieval-based), culminating in ChatGPT built on GPT-3/4 architectures.
• Chatbot cyber threats span client, network, response generation, and database modules, with multiple attack vectors and mitigations summarized.
• ChatGPT can be used to generate malware code, phishing emails, undetectable zero-day techniques, and LOLBINs under certain constraints, highlighting dual-use risks.
• Historical chatbot attacks and vulnerabilities illuminate ongoing challenges and the need for targeted countermeasures and governance.
• Future directions emphasize strategy, tooling, and policy measures to reduce harmful outcomes while preserving beneficial chatbot capabilities.