[论文解读] Deep PackGen: A Deep Reinforcement Learning Framework for Adversarial Network Packet Generation

Soumyadeep Hore, Jalal Ghadermazi+4 more|arXiv (Cornell University)|May 18, 2023

Adversarial Robustness in Machine Learning被引用 10

一句话总结

Deep PackGen 使用深度强化学习生成对抗性前向网络数据包，使其伪装成良性同时保持功能性，在代理 NIDS 分类器上的评估显示显著的对抗成功。

ABSTRACT

Recent advancements in artificial intelligence (AI) and machine learning (ML) algorithms, coupled with the availability of faster computing infrastructure, have enhanced the security posture of cybersecurity operations centers (defenders) through the development of ML-aided network intrusion detection systems (NIDS). Concurrently, the abilities of adversaries to evade security have also increased with the support of AI/ML models. Therefore, defenders need to proactively prepare for evasion attacks that exploit the detection mechanisms of NIDS. Recent studies have found that the perturbation of flow-based and packet-based features can deceive ML models, but these approaches have limitations. Perturbations made to the flow-based features are difficult to reverse-engineer, while samples generated with perturbations to the packet-based features are not playable. Our methodological framework, Deep PackGen, employs deep reinforcement learning to generate adversarial packets and aims to overcome the limitations of approaches in the literature. By taking raw malicious network packets as inputs and systematically making perturbations on them, Deep PackGen camouflages them as benign packets while still maintaining their functionality. In our experiments, using publicly available data, Deep PackGen achieved an average adversarial success rate of 66.4\% against various ML models and across different attack types. Our investigation also revealed that more than 45\% of the successful adversarial samples were out-of-distribution packets that evaded the decision boundaries of the classifiers. The knowledge gained from our study on the adversary's ability to make specific evasive perturbations to different types of malicious packets can help defenders enhance the robustness of their NIDS against evolving adversarial attacks.

研究动机与目标

• 开发基于 DRL 的框架，用于生成在保持数据包功能性的同时躲避基于 ML 的 NIDS 的对抗性网络数据包。
• 使用来自 PCAP 的原始前向数据包来训练数据包分类器并构建用于对抗性测试的集成代理模型。
• 评估对抗性扰动在不同分类器和环境中的鲁棒性与迁移性。
• 分析扰动策略及副作用，以提升防御方对日益演化的对抗攻击的鲁棒性。

提出的方法

• 处理原始 PCAP 数据，从 CICIDS-2017/2018 创建单向前向数据包数据集。
• 移除头部信息并归一化字节，以生成固定长度的数据包特征向量。
• 构建一个集成代理分类器来模拟防守方的 NIDS，并选择表现最佳的模型。
• 将对抗性数据包生成建模为带有受限扰动的序列决策问题（MDP）。
• 应用双 Q 学习（DDQN）DRL 来学习跨整个集成模型最大化规避奖励的扰动策略。
• 对未见过的分类器测试 DRL 生成的对抗性数据包，以衡量对抗成功率（ASR）。

实验结果

研究问题

• RQ1 DRL 智能体是否能够学习接近最优的对前向网络数据包的扰动，使基于 ML 的 NIDS 将恶意流量错误分类为良性？
• RQ2 扰动是否能保持数据包功能并在真实网络环境中实现？
• RQ3 在不同数据集和分类器之间，学习到的扰动策略的迁移性有多高？
• RQ4 有多少比例的成功对抗样本属于分布外（OOD），并且能够绕过复杂分类器的决策边界？

主要发现

• 基于 DRL 的 Deep PackGen 在多种机器学习模型和攻击类型下实现了平均 66.4% 的对抗成功率。
• 超过 45% 的成功对抗样本是分布外数据包，未越过分类器边界。
• 扰动仅限于前向数据包且为保持功能性的有效变更，已考虑副作用（例如校验和）。
• 该框架展示了学习到的扰动在不同网络环境中的可迁移性。
• 结果为在所提出约束下，哪些攻击类型更易被对手操控提供了洞见。