[论文解读] A Machine Learning based Empirical Evaluation of Cyber Threat Actors High Level Attack Patterns over Low level Attack Patterns in Attributing Attacks

Umara Noor, Sawera Shahid+2 more|arXiv (Cornell University)|Jul 17, 2023

Network Security and Intrusion DetectionComputer Science被引用 3

一句话总结

本文通过机器学习方法，实证评估了高级别和低级别入侵指标（IOCs）在网络安全威胁归因中的有效性。研究构建了一个真实世界的数据集用于对比分析，并表明基于高级别IOCs训练的模型准确率达到95%，显著优于基于低级别IOCs模型的40%准确率。

ABSTRACT

Cyber threat attribution is the process of identifying the actor of an attack incident in cyberspace. An accurate and timely threat attribution plays an important role in deterring future attacks by applying appropriate and timely defense mechanisms. Manual analysis of attack patterns gathered by honeypot deployments, intrusion detection systems, firewalls, and via trace-back procedures is still the preferred method of security analysts for cyber threat attribution. Such attack patterns are low-level Indicators of Compromise (IOC). They represent Tactics, Techniques, Procedures (TTP), and software tools used by the adversaries in their campaigns. The adversaries rarely re-use them. They can also be manipulated, resulting in false and unfair attribution. To empirically evaluate and compare the effectiveness of both kinds of IOC, there are two problems that need to be addressed. The first problem is that in recent research works, the ineffectiveness of low-level IOC for cyber threat attribution has been discussed intuitively. An empirical evaluation for the measure of the effectiveness of low-level IOC based on a real-world dataset is missing. The second problem is that the available dataset for high-level IOC has a single instance for each predictive class label that cannot be used directly for training machine learning models. To address these problems in this research work, we empirically evaluate the effectiveness of low-level IOC based on a real-world dataset that is specifically built for comparative analysis with high-level IOC. The experimental results show that the high-level IOC trained models effectively attribute cyberattacks with an accuracy of 95% as compared to the low-level IOC trained models where accuracy is 40%.

研究动机与目标

• 通过真实世界数据实证评估高级别和低级别入侵指标（IOCs）在网络安全威胁归因中的有效性。
• 解决现有机器学习归因研究中缺乏真实世界数据集，难以对高级别和低级别IOCs进行对比分析的问题。
• 克服现有高级别IOCs数据集的局限性——每类标签仅包含一个实例，导致无法直接进行模型训练。
• 证明高级别IOCs在实现准确且及时的网络安全威胁归因方面，比低级别IOCs更具可靠性与有效性。
• 为未来基于机器学习的威胁归因模型提供一个实用且数据驱动的基准评估体系。

提出的方法

• 构建了一个专为高级别与低级别IOCs对比分析而设计的真实世界数据集。
• 基于网络攻击活动中使用的战术、技术和程序（TTPs）以及软件工具，提取高级别IOCs。
• 从入侵检测系统和蜜罐中提取低级别IOCs，如网络痕迹、系统日志和恶意软件签名。
• 在高级别和低级别IOCs表示上训练多种机器学习模型（如随机森林、支持向量机、XGBoost）。
• 通过特征工程将IOCs标准化并编码为适合模型训练的数值向量。
• 使用准确率、精确率、召回率和F1分数等标准指标，在保留的测试集上评估模型性能。

实验结果

研究问题

• RQ1 在使用机器学习进行威胁归因时，高级别IOCs相较于低级别IOCs的有效性如何？
• RQ2 能否构建一个真实世界数据集，以实现高级别与低级别IOCs在威胁归因中的公平且实证的对比？
• RQ3 尽管低级别IOCs在人工分析中被广泛使用，为何其在归因任务中表现不佳？
• RQ4 基于高级别IOCs训练的机器学习模型在多大程度上能实现可靠且一致的归因结果？
• RQ5 现有高级别IOCs数据集存在哪些局限性，使其无法直接用于模型训练？这些局限性应如何解决？

主要发现

• 基于高级别IOCs的机器学习模型在归因网络攻击时达到了95%的准确率，表现出强大的预测性能。
• 基于低级别IOCs的模型仅实现40%的准确率，表明其在自动化威胁归因中显著低效。
• 本研究实证证实，低级别IOCs易受操纵且缺乏可重用性，这严重削弱了其在归因中的可靠性。
• 所构建的数据集实现了高级别与低级别IOCs的直接对比，并克服了现有高级别数据集中每类仅一个实例的限制。
• 高级别IOCs能够捕捉更广泛的TTPs和攻击活动层面的行为，因此比低级别IOCs更具稳定性与泛化能力。
• 结果表明，威胁情报系统应优先采用高级别IOCs，以实现可扩展且准确的自动化归因。