[Paper Review] On the model-checking-based IDS

Weijun Zhu|arXiv (Cornell University)|Jun 25, 2018

Network Security and Intrusion Detection6 references3 citations

TL;DR

This paper proposes the first standardized benchmark, ISIDMC, for evaluating model-checking-based intrusion detection systems (IDS) using Linear Temporal Logic (LTL), Interval Temporal Logic (ITL), and Real-time Attack Signature Logic (RASL). It evaluates detection accuracy and efficiency across 24 attack types, finding RASL-based detection most effective but slowest, while LTL-based detection is fastest but least expressive, confirming a trade-off between expressiveness and performance.

ABSTRACT

How to identify the comprehensive comparable performance of various Intrusion Detection (ID) algorithms which are based on the Model Checking (MC) techniques? To address this open issue, we conduct some tests for the model-checking-based intrusion detection systems (IDS) algorithms. At first, Linear Temporal Logic (LTL), Interval Temporal Logic (ITL) and Real-time Attack Signature Logic (RASL) are employed respectively to establish formula models for twenty-four types of attacks. And then, a standard intrusion set, called Intrusion Set for Intrusion Detection based on Model Checking (ISIDMC) is constructed. On the basis of it, detection abilities and efficiency of the intrusion detection algorithms based on model checking the three logics mentioned above are compared exhaustively

Motivation & Objective

• To address the lack of a standardized benchmark for evaluating model-checking-based intrusion detection systems (IDS) across different temporal logics.
• To compare the detection capabilities and execution efficiency of LTL-, ITL-, and RASL-based IDS algorithms on a comprehensive set of attack patterns.
• To establish a uniform evaluation framework for MC-based IDS, enabling fair performance comparison and guiding algorithm selection in real-world deployment.
• To empirically validate the trade-off between detection expressiveness and computational efficiency in model-checking-based intrusion detection.

Proposed method

• Formalizing 24 attack types using temporal logic formulas in LTL, ITL, and RASL to model complex behavioral relationships among attack actions.
• Constructing ISIDMC, a behavior-centric log dataset modeled after KDDCUP but tailored for model-checking-based IDS evaluation.
• Implementing model-checking algorithms based on LTL, ITL, and RASL to verify whether audit logs satisfy the defined attack formulas.
• Measuring detection performance via detection rate (true positives) and execution time (offline and online scenarios) across all attack types.
• Analyzing time complexity by examining formula structures, particularly the nesting depth of logical operators like ¬ and ;, to correlate with computational cost.
• Evaluating memory consumption via state space size to assess scalability and resource usage of each logic-based approach.

Experimental results

Research questions

• RQ1 How do LTL-, ITL-, and RASL-based IDS algorithms compare in detection accuracy across 24 types of network attacks?
• RQ2 What is the relationship between the expressive power of temporal logics and their detection performance in identifying complex attack behaviors?
• RQ3 How does the computational complexity of model checking vary across LTL, ITL, and RASL when detecting different attack patterns?
• RQ4 Is there a consistent trade-off between detection accuracy and execution time across the three model-checking-based IDS approaches?
• RQ5 Can a unified benchmark be established for evaluating model-checking-based IDS, and how does it differ from data-centric benchmarks like KDDCUP?

Key findings

• The RASL-based IDS achieved the highest detection rate across all attack types, followed by ITL-based and then LTL-based, due to RASL’s superior expressiveness in modeling temporal and real-time constraints.
• The LTL-based IDS showed the lowest detection rate but the fastest execution time, indicating a strong performance-efficiency trade-off.
• A clear inverse relationship exists between detection performance and time consumption: higher expressiveness (RASL) leads to higher accuracy but longer execution times.
• The worst-case time complexity for ITL- and RASL-based algorithms reaches O(2^{2n}) when detecting IP sweep and port scan attacks, due to deep nesting of logical operators.
• The LTL-based algorithm has a worst-case complexity of O(2^n), significantly lower than ITL and RASL, explaining its faster execution despite lower detection accuracy.
• State space growth remains manageable for all three approaches when modeling common DOS, probing, U2R, and R2L attacks, indicating feasible memory usage in practice.