[论文解读] Construction of Two Statistical Anomaly Features for Small-Sample APT Attack Traffic Classification

Ru Zhang, Wenxin Sun+4 more|arXiv (Cornell University)|Oct 27, 2020

Network Security and Intrusion Detection参考文献 61被引用 5

一句话总结

本文提出两种新型统计异常特征——C2Load_fluct（响应数据包负载波动）和Bad_rate（坏包率），以提升小样本环境下高级持续性威胁（APT）攻击流量的分类性能。通过将这些特征与现有特征结合，并采用改进的PADASYN数据平衡方法与AdaBoost算法，该方法在两个数据集上的F1得分分别达到0.98和0.94，显著提升了对APT C2流量的检测准确率。

ABSTRACT

Advanced Persistent Threat (APT) attack, also known as directed threat attack, refers to the continuous and effective attack activities carried out by an organization on a specific object. They are covert, persistent and targeted, which are difficult to capture by traditional intrusion detection system(IDS). The traffic generated by the APT organization, which is the organization that launch the APT attack, has a high similarity, especially in the Command and Control(C2) stage. The addition of features for APT organizations can effectively improve the accuracy of traffic detection for APT attacks. This paper analyzes the DNS and TCP traffic of the APT attack, and constructs two new features, C2Load_fluct (response packet load fluctuation) and Bad_rate (bad packet rate). The analysis showed APT attacks have obvious statistical laws in these two features. This article combines two new features with common features to classify APT attack traffic. Aiming at the problem of data loss and boundary samples, we improve the Adaptive Synthetic(ADASYN) Sampling Approach and propose the PADASYN algorithm to achieve data balance. A traffic classification scheme is designed based on the AdaBoost algorithm. Experiments show that the classification accuracy of APT attack traffic is improved after adding new features to the two datasets so that 10 DNS features, 11 TCP and HTTP/HTTPS features are used to construct a Features set. On the two datasets, F1-score can reach above 0.98 and 0.94 respectively, which proves that the two new features in this paper are effective for APT traffic detection.

研究动机与目标

• 解决传统入侵检测系统在小样本数据集上因高级持续性威胁（APT）攻击的隐蔽性与持久性而失效的问题。
• 识别DNS与TCP流量中的统计异常，以区分APT C2通信与正常流量。
• 通过引入两种专为APT流量模式设计的新统计特征，提升分类准确率。
• 利用改进的基于ADASYN的采样方法（PADASYN）克服APT流量数据集中的数据不平衡问题。
• 设计一种高效的流量分类框架，结合新特征与AdaBoost算法，实现高精度的APT检测。

提出的方法

• 构建C2Load_fluct作为衡量C2流量中响应数据包负载大小波动的指标，捕捉APT命令与控制通道特有的异常负载波动特征。
• 定义Bad_rate为APT流量中损坏或可疑数据包所占比例，识别与正常协议行为的偏离。
• 将上述两种新特征与10个DNS特征及11个TCP/HTTP/HTTPS特征结合，形成扩展的特征集用于分类。
• 提出PADASYN，一种改进的自适应采样技术，通过更好处理边界样本与小样本数据集中的数据丢失问题，优化ADASYN算法。
• 应用AdaBoost算法，在经平衡处理且特征丰富的数据集上训练鲁棒的集成分类器，用于APT流量检测。
• 在两个真实世界APT流量数据集上评估模型性能，验证其在小样本条件下的有效性。

实验结果

研究问题

• RQ1 数据包负载波动与坏包率的统计异常是否能有效区分APT C2流量与良性流量？
• RQ2 所提出的特征C2Load_fluct与Bad_rate在小样本APT检测场景下的表现如何？
• RQ3 与标准ADASYN相比，PADASYN算法在APT数据集中对数据平衡与分类性能的提升程度如何？
• RQ4 将两种新特征与现有特征结合后，对整体APT流量分类准确率的影响如何？
• RQ5 在结合PADASYN与新特征的前提下，基于AdaBoost的分类器是否能在数据不平衡的APT流量数据集中实现高F1得分？

主要发现

• C2Load_fluct与Bad_rate特征在APT流量中表现出显著的统计模式，尤其在C2阶段，可实现可靠检测。
• 引入两种新特征后，分类性能显著提升，在一个数据集上达到0.98的F1得分，在另一个数据集上达到0.94。
• PADASYN有效缓解了数据不平衡与边界样本问题，提升了小样本APT数据集上的模型泛化能力。
• 结合增强特征集的AdaBoost分类器优于基线模型，展现出在APT检测中强大的鲁棒性与高精度。
• 统计异常特征与改进采样方法的结合，在小样本APT流量分类任务中达到了当前最优性能。
• 结果证实，所提出的两种特征在训练数据有限的情况下，对识别APT C2流量具有高度有效性。