[Paper Review] Threat Detection and Investigation with System-level Provenance Graphs: A Survey

Zhenyuan Li, Qi Alfred Chen+2 more|arXiv (Cornell University)|Jun 2, 2020

Advanced Malware Detection Techniques75 references8 citations

TL;DR

This survey presents a comprehensive analysis of system-level provenance graphs for cyber threat detection and digital forensics, proposing a three-module architecture—data collection, data management, and threat detection—while identifying key challenges in scalability, false positives, and real-time performance. It provides a taxonomy of existing techniques and offers practical guidance for deploying provenance-based detection systems in real-world environments.

ABSTRACT

With the development of information technology, the border of the cyberspace gets much broader, exposing more and more vulnerabilities to attackers. Traditional mitigation-based defence strategies are challenging to cope with the current complicated situation. Security practitioners urgently need better tools to describe and modelling attacks for defence. The provenance graph seems like an ideal method for threat modelling with powerful semantic expression ability and attacks historic correlation ability. In this paper, we firstly introduce the basic concepts about system-level provenance graph and proposed typical system architecture for provenance graph-based threat detection and investigation. A comprehensive provenance graph-based threat detection system can be divided into three modules, namely, "data collection module", "data management module", and "threat detection modules". Each module contains several components and involves many research problem. We systematically analyzed the algorithms and design details involved. By comparison, we give the strategy of technology selection. Moreover, we pointed out the shortcomings of the existing work for future improvement.

Motivation & Objective

• To address the limitations of traditional mitigation-based defenses in detecting advanced, stealthy attacks like APTs.
• To establish provenance graphs as a robust, semantically rich method for modeling system behavior and attack history.
• To systematize the design of provenance graph-based detection systems through a modular architecture and comparative analysis.
• To identify practical challenges such as false positives, lack of standardized datasets, and evasion resistance.
• To guide future research by outlining key technical trade-offs and deployment strategies for real-world systems.

Proposed method

• Proposes a three-module system architecture: data collection, data management, and threat detection, each with specialized components.
• Employs a taxonomy of existing techniques, categorizing approaches by data collection methods, storage optimizations, and detection algorithms.
• Uses performance metrics such as detection accuracy, false positive rate, and response time to compare 40+ existing systems.
• Analyzes sequence-based and graph-based detection models, highlighting trade-offs between efficiency and robustness.
• Evaluates real-time processing strategies, including tag propagation and alert correlation, to balance low latency and high accuracy.
• Recommends a hybrid approach combining lightweight real-time detection with graph-based analysis for improved robustness.

Experimental results

Research questions

• RQ1 How can provenance graph storage be optimized for space efficiency without losing semantic fidelity?
• RQ2 How can query performance be balanced with storage efficiency in large-scale provenance graph systems?
• RQ3 How can detection models achieve high true-positive rates while minimizing false positives?
• RQ4 What techniques can reduce response time in real-time threat detection and forensics?
• RQ5 How can provenance graph-based systems resist evasion and ensure reliability in production environments?

Key findings

• Sequence-based detection models like tag propagation offer lower latency and are better suited for real-time deployment, while graph-based models provide higher accuracy and robustness.
• False positive rates remain a major challenge, with most systems relying on empirical parameter tuning rather than principled optimization.
• The only publicly available benchmark dataset for provenance-based detection is the DARPA Transparent Computing Engagement 3 and 5 datasets, which are limited in scale and diversity.
• Existing systems lack systematic evaluation of evasion resistance, leaving a critical gap in threat model reliability.
• A hybrid detection approach combining lightweight real-time analysis with graph-based correlation offers the best trade-off between speed and accuracy.
• There is a strong need for standardized datasets and data formats to enable reproducibility and quantitative comparison across systems.