[论文解读] AIQL: Enabling Efficient Attack Investigation from System Monitoring Data

Peng Gao, Xusheng Xiao+5 more|arXiv (Cornell University)|Jun 6, 2018

Network Security and Intrusion Detection参考文献 28被引用 40

一句话总结

AIQL 引入了领域特定的查询语言以及优化的存储/执行引擎，使对大规模系统监控数据的快速、简洁的攻击调查成为可能，在性能上远超基于 SQL 的和基于图的系统，达到数量级的提升。

ABSTRACT

The need for countering Advanced Persistent Threat (APT) attacks has led to the solutions that ubiquitously monitor system activities in each host, and perform timely attack investigation over the monitoring data for analyzing attack provenance. However, existing query systems based on relational databases and graph databases lack language constructs to express key properties of major attack behaviors, and often execute queries inefficiently since their semantics-agnostic design cannot exploit the properties of system monitoring data to speed up query execution. To address this problem, we propose a novel query system built on top of existing monitoring tools and databases, which is designed with novel types of optimizations to support timely attack investigation. Our system provides (1) domain-specific data model and storage for scaling the storage, (2) a domain-specific query language, Attack Investigation Query Language (AIQL) that integrates critical primitives for attack investigation, and (3) an optimized query engine based on the characteristics of the data and the semantics of the queries to efficiently schedule the query execution. We deployed our system in NEC Labs America comprising 150 hosts and evaluated it using 857 GB of real system monitoring data (containing 2.5 billion events). Our evaluations on a real-world APT attack and a broad set of attack behaviors show that our system surpasses existing systems in both efficiency (124x over PostgreSQL, 157x over Neo4j, and 16x over Greenplum) and conciseness (SQL, Neo4j Cypher, and Splunk SPL contain at least 2.4x more constraints than AIQL).

研究动机与目标

• 推动对大规模系统监控数据的及时攻击调查，以应对高级持续性威胁（APTs）。
• 提出一个领域特定的数据模型和存储策略，以扩展监控数据规模。
• 设计 Aiql，一种用于攻击行为的领域特定查询语言。
• 开发一个针对系统监控数据的优化、专用的查询引擎。
• 在真实世界数据上评估 Aiql 相对于传统的 SQL/图数据库系统。
• method1
• method2

提出的方法

• 提出一个领域特定的数据模型和存储方案，将数据按时间和空间分区，以实现并行处理。
• 引入 Aiql，具有多事件模式、依赖跟踪和异常查询的结构。
• 开发一个查询引擎，为每个事件模式综合 SQL 查询，并使用面向关系的调度。
• 实现时间窗口分区并在 MPP 数据库（如 Greenplum）上分布执行。
• 使用两层优化：(a) 事件模式剪枝能力，(b) 时空并行化以并行执行子查询。
• 提供上下文感知的语法简写和属性推断，以简化查询编写。

实验结果

研究问题

• RQ1 Can Aiql express key attack behaviors (multi-step attacks, dependencies, anomalies) succinctly and effectively?
• RQ2 Do domain-specific optimizations in storage and query scheduling substantially improve attack-investigation performance on large monitoring datasets?
• RQ3 How does Aiql compare to SQL, Cypher, and Splunk SPL in efficiency and conciseness for typical APT-related queries?
• RQ4 Can the system scale to enterprise-scale deployments with hundreds of hosts and billions of events?

主要发现

• Aiql achieves significant end-to-end efficiency gains, surpassing PostgreSQL by 124x and Neo4j by 157x on real attack scenarios.
• Scheduling with relationship-based pruning and temporal/spatial parallelization yields 40x faster scheduling over PostgreSQL and 16x over Greenplum.
• Aiql queries are more concise than SQL, Neo4j Cypher, and Splunk SPL, requiring at least 2.4x fewer constraints, 3.1x fewer words, and 4.7x fewer characters.
• The evaluation used 857 GB of real system monitoring data (16 days, 2.5 billion events) across 150 hosts, demonstrating practicality for enterprise-scale deployments.
• Aiql supports three attack-behavior primitives (multi-step, dependency tracking, and anomaly-based queries) and integrates domain-specific data modeling to optimize storage and query execution.