[论文解读] Evaluation of Machine Learning Classifiers for Zero-Day Intrusion Detection -- An Analysis on CIC-AWS-2018 dataset

Qianru Zhou, Dimitrios P. Pezaros|arXiv (Cornell University)|May 9, 2019

Network Security and Intrusion Detection参考文献 9被引用 52

一句话总结

本文在 CIC-AWS-2018 数据集上对六种 ML 分类器进行零日入侵检测评估，使用 CICFlowMeter 特征，发现像决策树等简单模型也能达到较高的准确率。

ABSTRACT

Detecting Zero-Day intrusions has been the goal of Cybersecurity, especially intrusion detection for a long time. Machine learning is believed to be the promising methodology to solve that problem, numerous models have been proposed but a practical solution is still yet to come, mainly due to the limitation caused by the out-of-date open datasets available. In this paper, we take a deep inspection of the flow-based statistical data generated by CICFlowMeter, with six most popular machine learning classification models for Zero-Day attacks detection. The training dataset CIC-AWS-2018 Dataset contains fourteen types of intrusions, while the testing datasets contains eight different types of attacks. The six classification models are evaluated and cross validated on CIC-AWS-2018 Dataset for their accuracy in terms of false-positive rate, true-positive rate, and time overhead. Testing dataset, including eight novel (or Zero-Day) real-life attacks and benign traffic flows collected in real research production network are used to test the performance of the chosen decision tree classifier. Promising results are received with the accuracy as high as 100% and reasonable time overhead. We argue that with the statistical data collected from CICFlowMeter, simple machine learning models such as the decision tree classification could be able to take charge in detecting Zero-Day attacks.

研究动机与目标

• 使用 CICFlowMeter 的基于流的统计数据来评估零日入侵检测性能。
• 在 CIC-AWS-2018 的训练/测试拆分上比较六种机器学习分类器。
• 分析每个模型的准确率、误报/真正率以及时间开销。
• 确定简单模型是否能有效检测零日攻击。

提出的方法

• 使用 CICFlowMeter 从网络数据中提取基于流的统计特征。
• 在带有14种训练入侵类型和8种测试类型的 CIC-AWS-2018 数据集上评估六个 ML 分类模型。
• 进行交叉验证以测量准确率、误报率、真正率和时间开销。
• 在真实生活的零日攻击和良性流量上测试所选模型，以评估实际性能。
• 比较结果以确定简单模型（如决策树）是否适用于零日检测。

实验结果

研究问题

• RQ1 在 CIC-AWS-2018 上，哪些 ML 分类器在准确性和开销之间达到最佳平衡，以实现零日入侵检测？
• RQ2 简单模型如决策树在该数据集上是否能与更复杂的模型相比拟？
• RQ3 模型在实际生产网络数据中的零日攻击和良性流量的表现如何？

主要发现

• 在 CIC-AWS-2018 数据集上评估了六种 ML 模型。
• 训练集包含十四种入侵类型；测试集包含八种入侵类型。
• 使用真实零日攻击和良性流量进行测试，结果显示准确率高达 100% 的前景。
• 模型的时间开销保持在合理范围，支持实际部署。
• 研究认为，简单模型如决策树在使用 CICFlowMeter 特征时可以有效检测零日攻击。