[Paper Review] Discovering general partial orders in event streams

Avinash Achar, Srivatsan Laxman+2 more|ArXiv.org|Feb 7, 2009

Data Mining Algorithms and Applications17 references3 citations

TL;DR

This paper presents efficient algorithms for discovering frequent episodes with general partial orders in event streams, using finite state automata to track non-overlapping occurrences of injective episodes. It introduces bidirectional evidence as a novel interestingness measure to filter out combinatorially explosive, uninteresting patterns, demonstrating scalability and effectiveness through extensive simulations on synthetic data streams with varying noise and pattern complexity.

ABSTRACT

Frequent episode discovery is a popular framework for pattern discovery in event streams. An episode is a partially ordered set of nodes with each node associated with an event type. Efficient (and separate) algorithms exist for episode discovery when the associated partial order is total (serial episode) and trivial (parallel episode). In this paper, we propose efficient algorithms for discovering frequent episodes with general partial orders. These algorithms can be easily specialized to discover serial or parallel episodes. Also, the algorithms are flexible enough to be specialized for mining in the space of certain interesting subclasses of partial orders. We point out that there is an inherent combinatorial explosion in frequent partial order mining and most importantly, frequency alone is not a sufficient measure of interestingness. We propose a new interestingness measure for general partial order episodes and a discovery method based on this measure, for filtering out uninteresting partial orders. Simulations demonstrate the effectiveness of our algorithms.

Motivation & Objective

• To address the lack of efficient algorithms for discovering frequent episodes with general partial orders in event streams.
• To overcome the combinatorial explosion in the number of frequent episodes when using frequency alone as a measure of interestingness.
• To develop a flexible framework that can specialize to mine serial, parallel, or custom subclasses of partial order episodes.
• To introduce and validate a new interestingness measure—bidirectional evidence—that captures relative frequency patterns between event pairs.
• To demonstrate scalability and effectiveness of the proposed method across varying data lengths, noise levels, and embedded pattern counts.

Proposed method

• Uses finite state automata (FSA) to efficiently track non-overlapping occurrences of injective episodes in a single event stream.
• Employs a candidate generation strategy that supports specialization to subclasses of partial orders with the maximal subepisode property, including serial and parallel episodes.
• Introduces a new interestingness measure called bidirectional evidence, defined as the entropy-like divergence between forward and reverse co-occurrence frequencies of event pairs in episode occurrences.
• Applies expiry-time constraints to limit the time span of valid episode occurrences, ensuring temporal relevance.
• Designs a two-threshold mining framework: one for minimum frequency and another for minimum bidirectional evidence to filter uninteresting patterns.
• Optimizes the algorithm for scalability by leveraging the injective episode constraint, which prevents event-type repetition within a single episode.

Experimental results

Research questions

• RQ1 How can frequent episodes with general partial orders be efficiently discovered in a single, long event stream, given the absence of existing algorithms for this case?
• RQ2 What causes the combinatorial explosion in frequent partial order mining, and how can it be mitigated beyond relying solely on frequency thresholds?
• RQ3 Can a new measure of interestingness be defined that captures structural and temporal diversity in episode patterns, especially regarding event order reversal?
• RQ4 How does the proposed bidirectional evidence measure improve the selection of meaningful partial order episodes compared to frequency alone?
• RQ5 To what extent does the algorithm scale with data length, noise level, and number of embedded patterns in realistic synthetic event streams?

Key findings

• The algorithm scales linearly with data length, with runtimes increasing from 52 seconds at 22,500 events to 3 minutes 25 seconds at 90,000 events, maintaining a constant fth/T ratio.
• At a noise level of 0.88 (ρ = 0.045), runtimes peak at 11 minutes 10 seconds due to a surge in 3-node candidates, highlighting sensitivity to noise.
• With 8 embedded 8-node episodes, the number of frequent episodes drops from 100 to 10 as the bidirectional evidence threshold (H(α)) increases from 0.2 to 1.0, showing effective filtering.
• For low noise (ρ = 0.02), runtimes remain minimal because only signal events are frequent at the 1-node level, reducing candidate generation.
• The method successfully discovers meaningful partial order episodes even in high-noise environments, with bidirectional evidence effectively eliminating redundant or uninformative patterns.
• The algorithm can be specialized to mine only serial or only parallel episodes, demonstrating its generality and unification of existing approaches.