[论文解读] Beyond Free Riding: Quality of Indicators for Assessing Participation in Information Sharing for Threat Intelligence

Omar Alibrahim, Aziz Mohaisen+3 more|arXiv (Cornell University)|Feb 2, 2017

Information and Cyber Security参考文献 27被引用 12

一句话总结

本文提出指标质量（QoI）作为多维度指标，用于评估威胁情报共享中参与者的真正贡献，超越简单的数量度量。通过利用真实世界的杀毒软件扫描数据，评估指标的正确性、相关性、实用性和独特性，本研究证明QoI能有效识别出体积度量所遗漏的搭便车行为，揭示出高数量贡献者往往提供低质量的指标。

ABSTRACT

Threat intelligence sharing has become a growing concept, whereby entities can exchange patterns of threats with each other, in the form of indicators, to a community of trust for threat analysis and incident response. However, sharing threat-related information have posed various risks to an organization that pertains to its security, privacy, and competitiveness. Given the coinciding benefits and risks of threat information sharing, some entities have adopted an elusive behavior of "free-riding" so that they can acquire the benefits of sharing without contributing much to the community. So far, understanding the effectiveness of sharing has been viewed from the perspective of the amount of information exchanged as opposed to its quality. In this paper, we introduce the notion of quality of indicators (\qoi) for the assessment of the level of contribution by participants in information sharing for threat intelligence. We exemplify this notion through various metrics, including correctness, relevance, utility, and uniqueness of indicators. In order to realize the notion of \qoi, we conducted an empirical study and taken a benchmark approach to define quality metrics, then we obtained a reference dataset and utilized tools from the machine learning literature for quality assessment. We compared these results against a model that only considers the volume of information as a metric for contribution, and unveiled various interesting observations, including the ability to spot low quality contributions that are synonym to free riding in threat information sharing.

研究动机与目标

• 解决威胁情报共享社区中缺乏质量意识的贡献评估问题。
• 识别并量化体积度量无法检测到的搭便车行为。
• 开发一种稳健且上下文敏感的框架，用于衡量共享威胁指标的实际价值。
• 确立QoI作为评估威胁情报生态系统中参与者实用性的优于体积度量的替代方案。

提出的方法

• 将QoI定义为综合指标，涵盖威胁指标的正确性、相关性、实用性和独特性。
• 收集了经人工验证的恶意软件样本作为质量评估的基准数据集。
• 应用机器学习技术，基于质量维度对指标进行评估与打分。
• 使用真实世界的杀毒软件扫描数据，将QoI指标与基于体积的贡献模型进行基准对比。
• 通过对比分析，将QoI评分与多个厂商的基于体积的评分进行对比。
• 识别高数量与高质量贡献之间的差异，以检测潜在的搭便车行为。

实验结果

研究问题

• RQ1 如何在不依赖简单数量统计的前提下，对威胁指标的质量进行定量测量？
• RQ2 基于体积的贡献度量在多大程度上无法检测到威胁情报共享中的搭便车行为？
• RQ3 指标的正确性、相关性、实用性和独特性如何与实际贡献价值相关联？
• RQ4 QoI指标能否区分高数量但低质量的贡献者与真正有价值的贡献者？
• RQ5 上下文相关的质量标记对去中心化威胁情报系统中贡献评估准确性有何影响？

主要发现

• 基于体积的贡献度量无法检测到搭便车行为，因为部分高数量贡献者表现出接近零的QoI得分。
• 如vendor 11、vendor 18和vendor 20等厂商虽贡献了大量指标，但QoI得分极低，表明可能存在搭便车行为。
• 基于正确性、相关性和实用性的QoI指标与基于体积的评分存在显著差异，证明高数量并不意味着高质量。
• QoI框架成功识别出仅凭体积无法区分的低质量贡献，这些贡献与搭便车行为无异。
• QoI指标提供了比基于体积的方法更稳健、更具上下文敏感性且更具可操作性的贡献度量方式。
• 本研究证实，QoI能够捕捉到超越单纯数据量的有意义贡献，尤其在去中心化的威胁情报社区中表现突出。