[论文解读] HuntGPT: Integrating Machine Learning-Based Anomaly Detection and Explainable AI with Large Language Models (LLMs)

Tarek Ali, Panos Kostakos|arXiv (Cornell University)|Sep 27, 2023

Network Security and Intrusion Detection被引用 39

一句话总结

HuntGPT 将基于随机森林的网络异常检测器与 KDD99 训练，结合 XAI（SHAP/LIME）以及 OpenAI GPT-3.5 Turbo 聊天机器人，通过 Gradio 仪表板提供可解释、可执行的入侵检测洞察。该研究评估 AI 辅助系统的技术准确性和响应可读性。

ABSTRACT

Machine learning (ML) is crucial in network anomaly detection for proactive threat hunting, reducing detection and response times significantly. However, challenges in model training, maintenance, and frequent false positives impact its acceptance and reliability. Explainable AI (XAI) attempts to mitigate these issues, allowing cybersecurity teams to assess AI-generated alerts with confidence, but has seen limited acceptance from incident responders. Large Language Models (LLMs) present a solution through discerning patterns in extensive information and adapting to different functional requirements. We present HuntGPT, a specialized intrusion detection dashboard applying a Random Forest classifier using the KDD99 dataset, integrating XAI frameworks like SHAP and Lime for user-friendly and intuitive model interaction, and combined with a GPT-3.5 Turbo, it delivers threats in an understandable format. The paper delves into the system's architecture, components, and technical accuracy, assessed through Certified Information Security Manager (CISM) Practice Exams, evaluating response quality across six metrics. The results demonstrate that conversational agents, supported by LLM and integrated with XAI, provide robust, explainable, and actionable AI solutions in intrusion detection, enhancing user understanding and interactive experience.

研究动机与目标

• Motivate the use of ML-based anomaly detection in threat hunting to reduce detection time and improve response quality.
• Propose a dashboard (HuntGPT) that integrates a Random Forest detector with SHAP and LIME explanations and a GPT-based conversational agent.
• Enhance analyst trust and usability by combining explainability with interactive AI-assisted analysis.
• Evaluate the prototype's technical accuracy and readability of AI-generated explanations and responses.

提出的方法

• Train and deploy a Random Forest classifier on the KDD99 intrusion detection dataset for anomaly detection.
• Integrate SHAP and LIME explainability frameworks to generate feature-level explanations and visual plots stored in Elasticsearch and AWS S3.
• Attach a GPT-3.5 Turbo conversational agent via OpenAI API to deliver explainable threat analyses through the IDS dashboard.
• Use a three-layer architecture (Analytics engine, Data Storage with Elasticsearch, and UI with Gradio) to separate concerns and enable modular development.
• Evaluate technical accuracy against cybersecurity certification prep materials (CISM) and assess readability of AI explanations using six readability metrics.

Figure 1: High level diagram of dashboard integration.

实验结果

研究问题

• RQ1 Can an LLM-powered chatbot paired with XAI explanations provide accurate and actionable insights for detected network anomalies?
• RQ2 Does integrating SHAP and LIME explanations improve analyst trust and understanding of ML-based anomaly detections?
• RQ3 What is the cognitive readability of AI-generated anomaly explanations and chatbot responses in a cybersecurity context?
• RQ4 How well does the system perform in terms of practical cybersecurity knowledge as measured against standard certifications?

主要发现

Exam	No. of Questions	GPT-3.5 turbo Success Rate
CISM Certified Information Security Manager Practice Exams [14]	40	82.5%
ISACA official CISM practice Quiz [43]	10	80%
ISACA official cybersecurity fundamentals practice quiz [43]	25	72%

• GPT-3.5 Turbo demonstrated substantial cybersecurity knowledge with success rates of 72% to 82.5% on standardized exams used in the study.
• Readability analyses indicate generated explanations and chatbot responses are generally at a graduate or equivalent level, yet remain comprehensible to users with basic college education.
• The HuntGPT prototype delivers explainable anomaly detections via AI-generated explanations, interactive discussions, and a downloadable incident report.
• The architecture supports modular development with Elasticsearch storage, AWS S3 plots, and a Gradio UI integrated with OpenAI’s API for seamless analyst interaction.
• Conversational agent–driven explanations can aid in generating actionable AI-supported responses within intrusion detection contexts.

Figure 2: System Components Diagram