[论文解读] Revolutionizing Cyber Threat Detection with Large Language Models: A privacy-preserving BERT-based Lightweight Model for IoT/IIoT Devices

Mohamed Amine Ferrag, Mthandazo Ndhlovu+5 more|arXiv (Cornell University)|Jun 25, 2023

Network Security and Intrusion Detection被引用 22

一句话总结

SecurityBERT 是一种隐私保护、轻量级的 15‑layer 基于 BERT 的模型，用于 IoT/IIoT 网络威胁检测，使用 PPFLLE 编码和 BBPE，能够在资源受限设备上实现高准确性和快速推理。

ABSTRACT

The field of Natural Language Processing (NLP) is currently undergoing a revolutionary transformation driven by the power of pre-trained Large Language Models (LLMs) based on groundbreaking Transformer architectures. As the frequency and diversity of cybersecurity attacks continue to rise, the importance of incident detection has significantly increased. IoT devices are expanding rapidly, resulting in a growing need for efficient techniques to autonomously identify network-based attacks in IoT networks with both high precision and minimal computational requirements. This paper presents SecurityBERT, a novel architecture that leverages the Bidirectional Encoder Representations from Transformers (BERT) model for cyber threat detection in IoT networks. During the training of SecurityBERT, we incorporated a novel privacy-preserving encoding technique called Privacy-Preserving Fixed-Length Encoding (PPFLE). We effectively represented network traffic data in a structured format by combining PPFLE with the Byte-level Byte-Pair Encoder (BBPE) Tokenizer. Our research demonstrates that SecurityBERT outperforms traditional Machine Learning (ML) and Deep Learning (DL) methods, such as Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs), in cyber threat detection. Employing the Edge-IIoTset cybersecurity dataset, our experimental analysis shows that SecurityBERT achieved an impressive 98.2% overall accuracy in identifying fourteen distinct attack types, surpassing previous records set by hybrid solutions such as GAN-Transformer-based architectures and CNN-LSTM models. With an inference time of less than 0.15 seconds on an average CPU and a compact model size of just 16.7MB, SecurityBERT is ideally suited for real-life traffic analysis and a suitable choice for deployment on resource-constrained IoT devices.

研究动机与目标

• 解决在资源受限的 IoT/IIoT 环境下实现准确的网络威胁检测的需求。
• 开发隐私保护的数据编码，以便在网络流量数据上使 NLP 模型可用。
• 创建适合在设备端或边缘部署的轻量级基于 BERT 的架构。
• 在一个现实的 IoT/IIoT 数据集上展示相对于传统 ML/DL 方法的卓越性能。

提出的方法

• 引入 Privacy-Preserving Fixed-Length Encoding (PPFLE)，将网络特征转换为文本样表示。
• 将 PPFLE 与 Byte-level BPE (BBPE) 分词器结合，以使用 transformer 处理编码数据。
• 设计一个 15-layer、11M 参数的 SecurityBERT 模型，在 PPFLE-编码数据上进行预训练。
• 通过带自注意力的 transformer 编码器应用上下文表征，用于多类别威胁分类。
• 在 Edge-IIoTset 数据集上进行评估，并与 CNNs 和基于 LLM 的方法进行比较。

实验结果

研究问题

• RQ1 一种隐私保护的定长编码方案是否能够在 IoT/IIoT 流量上实现有效的基于 BERT 的网络威胁检测？
• RQ2 使用 PPFLE-BBPE 的轻量级 BERT 架构是否在边缘设备上实现比传统 ML/DL 模型更高的准确性？
• RQ3 在典型的 CPU/GPU 硬件上进行实时流量分析时，SecurityBERT 的推理时间和内存需求是多少？

主要发现

• SecurityBERT 在 Edge-IIoTset 上覆盖十四种攻击类型的总体准确率达到 98.2%。
• 推理时间在平均 CPU 硬件上小于 0.15 秒。
• 模型大小为 16.7 MB，参数为 11M，适用于资源受限设备。
• PPFLE 编码 + BBPE 分词器在传统 ML 和 DL 基线（包括 CNNs 和 LSTM 基于的模型）之上表现更好。
• 该方法通过使用散列文本表示而非原始网络数据来维持隐私。