[Paper Review] Mining Periodic Patterns with a MDL Criterion

Esther Galbrun, Peggy Cellier+3 more|arXiv (Cornell University)|Jul 4, 2018

Data Mining Algorithms and Applications18 references4 citations

TL;DR

This paper proposes a novel approach for mining periodic patterns in event logs using the Minimum Description Length (MDL) principle to evaluate and select pattern sets that best compress the data. By defining an expressive pattern language with nested cycles over events and using MDL-based encoding to balance model complexity and data fit, the method effectively identifies meaningful periodic structures in real-world logs, achieving significant compression and revealing intuitive, recurring patterns.

ABSTRACT

The quantity of event logs available is increasing rapidly, be they produced by industrial processes, computing systems, or life tracking, for instance. It is thus important to design effective ways to uncover the information they contain. Because event logs often record repetitive phenomena, mining periodic patterns is especially relevant when considering such data. Indeed, capturing such regularities is instrumental in providing condensed representations of the event sequences. We present an approach for mining periodic patterns from event logs while relying on a Minimum Description Length (MDL) criterion to evaluate candidate patterns. Our goal is to extract a set of patterns that suitably characterises the periodic structure present in the data. We evaluate the interest of our approach on several real-world event log datasets.

Motivation & Objective

• To address the challenge of extracting meaningful periodic patterns from large, noisy event logs that contain mixed signals and irregularities.
• To reduce the explosion of patterns typically produced by traditional pattern mining algorithms by focusing on a globally optimal set of patterns.
• To leverage the MDL principle as a principled criterion for evaluating and selecting pattern sets that best describe the periodic structure in event sequences.
• To design an expressive pattern language capable of representing nested and composite periodic behaviors, such as daily routines with weekly repetitions.
• To develop and empirically validate an algorithm that efficiently mines MDL-optimal periodic patterns from diverse real-world event log datasets.

Proposed method

• Defining a pattern language based on cycles over single events, where each cycle is defined by a period, a recurrence count, and a start time.
• Extending the pattern language to allow recursive composition of patterns, enabling nested and composite periodic structures (e.g., daily routines repeated weekly).
• Designing an encoding scheme that assigns a code length to a pattern set based on how well it compresses the input sequence, using the MDL principle.
• Using a two-stage algorithm: first mining simple cycles from the data, then combining them into more complex patterns while evaluating MDL scores to ensure compression gain.
• Applying a greedy, iterative combination strategy that tests pattern combinations and retains only those that reduce the total code length.
• Implementing a pruning mechanism that discards candidate combinations that do not improve compression, ensuring scalability and efficiency.

Experimental results

Research questions

• RQ1 Can the MDL principle be effectively applied to select a compact, high-quality set of periodic patterns that capture the underlying structure in event logs?
• RQ2 How can a pattern language be designed to express complex, nested periodic behaviors such as daily routines repeated weekly?
• RQ3 To what extent can MDL-based pattern selection reduce the number of patterns while maintaining or improving data compression compared to traditional mining approaches?
• RQ4 How does the algorithm perform on real-world event logs with varying levels of noise, irregularity, and complexity?
• RQ5 What kinds of meaningful periodic patterns—especially nested or composite ones—can be reliably extracted from diverse log datasets?

Key findings

• The proposed MDL-based approach successfully identifies sets of periodic patterns that achieve significant data compression on real-world event log datasets, demonstrating the effectiveness of the criterion in selecting meaningful patterns.
• The algorithm extracts intuitive, human-interpretable patterns such as daily routines (e.g., waking up and preparing coffee every 24 hours) and system-level patterns (e.g., repeated context switches and process activations).
• Running times vary widely, from seconds to several hours, with the second stage (pattern combination) being the most time-consuming, particularly when many long cycles are extracted initially.
• The method is robust to noise and irregularities, such as gaps in recurrence and variable inter-occurrence delays, due to the flexibility of the pattern language and MDL evaluation.
• The algorithm’s performance is sensitive to the number and length of initial cycles, with longer and more numerous cycles increasing the computational cost of combination rounds.
• The approach outperforms traditional mining by reducing the number of patterns while maintaining or improving compression, as evidenced by the achieved compression ratios and code length reductions.