[论文解读] SAQL: A Stream-based Query System for Real-Time Abnormal System Behavior Detection

Peng Gao, Xusheng Xiao+7 more|arXiv (Cornell University)|Jun 25, 2018

Anomaly Detection Techniques and Applications参考文献 46被引用 55

一句话总结

SAQL 提供一个基于流的查询系统，旨在实现对异常系统行为的实时检测。

ABSTRACT

Recently, advanced cyber attacks, which consist of a sequence of steps that involve many vulnerabilities and hosts, compromise the security of many well-protected businesses. This has led to the solutions that ubiquitously monitor system activities in each host (big data) as a series of events, and search for anomalies (abnormal behaviors) for triaging risky events. Since fighting against these attacks is a time-critical mission to prevent further damage, these solutions face challenges in incorporating expert knowledge to perform timely anomaly detection over the large-scale provenance data. To address these challenges, we propose a novel stream-based query system that takes as input, a real-time event feed aggregated from multiple hosts in an enterprise, and provides an anomaly query engine that queries the event feed to identify abnormal behaviors based on the specified anomalies. To facilitate the task of expressing anomalies based on expert knowledge, our system provides a domain-specific query language, SAQL, which allows analysts to express models for (1) rule-based anomalies, (2) time-series anomalies, (3) invariant-based anomalies, and (4) outlier-based anomalies. We deployed our system in NEC Labs America comprising 150 hosts and evaluated it using 1.1TB of real system monitoring data (containing 3.3 billion events). Our evaluations on a broad set of attack behaviors and micro-benchmarks show that our system has a low detection latency (<2s) and a high system throughput (110,000 events/s; supporting ~4000 hosts), and is more efficient in memory utilization than the existing stream-based complex event processing systems.

研究动机与目标

• 动机：说明对实时检测异常系统行为的需求。
• 提出一个基于流的查询系统（SAQL），在数据到达时检测异常。
• 描述 SAQL 在实时监控中的设计、实现及潜在好处。

提出的方法

• 将 SAQL 介绍为用于异常行为检测的基于流的查询引擎。
• 描述用于实时处理的数据模型和流操作符。
• 解释如何在连续数据流中表达查询以检测异常。
• 讨论体系结构组件以及与流平台的集成。

实验结果

研究问题

• RQ1 流式查询系统如何实现对异常系统行为的实时检测？
• RQ2 SAQL 的设计原则与组件有哪些，使其能够在流上高效地检测异常？
• RQ3 在表达力、延迟和可扩展性方面，SAQL 与其他实时异常检测的流处理方法相比如何？

主要发现

• SAQL 通过基于流的查询方法实现对异常系统行为的实时检测。
• 该系统被设计为处理流数据并在事件到达时检测异常。
• SAQL 的设计强调用于在流式环境中进行持续异常检测的基于查询的接口。