[Paper Review] Log Skeletons: A Classification Approach to Process Discovery

H. M. W. Verbeek, Renata Medeiros de Carvalho|arXiv (Cornell University)|Jun 21, 2018

Business Process Modeling and AnalysisBusiness, Management and Accounting4 citations

TL;DR

This paper introduces log skeletons—a rule-based, constraint-driven model closely related to Declare—designed for process discovery via classification. Using an automated algorithm to construct log skeletons from event logs, the approach classifies traces with 194 out of 200 correct on the 2017 Process Discovery Contest, significantly outperforming state-of-the-art discovery algorithms like Inductive Miner, which achieved only 147 correct classifications.

ABSTRACT

To test the effectiveness of process discovery algorithms, a Process Discovery Contest (PDC) has been set up. This PDC uses a classification approach to measure this effectiveness: The better the discovered model can classify whether or not a new trace conforms to the event log, the better the discovery algorithm is supposed to be. Unfortunately, even the state-of-the-art fully-automated discovery algorithms score poorly on this classification. Even the best of these algorithms, the Inductive Miner, scored only 147 correct classified traces out of 200 traces on the PDC of 2017. This paper introduces the rule-based log skeleton model, which is closely related to the Declare constraint model, together with a way to classify traces using this model. This classification using log skeletons is shown to score better on the PDC of 2017 than state-of-the-art discovery algorithms: 194 out of 200. As a result, one can argue that the fully-automated algorithm to construct (or: discover) a log skeleton from an event log outperforms existing state-of-the-art fully-automated discovery algorithms.

Motivation & Objective

• To improve process discovery by reframing it as a classification problem, where model quality is measured by trace classification accuracy.
• To address the limitation of existing fully-automated discovery algorithms in correctly classifying traces from event logs.
• To propose a novel, automated method for constructing log skeletons—rule-based models capturing structural constraints in event logs.
• To demonstrate that a classification-based approach using log skeletons can surpass traditional discovery algorithms in performance.
• To enable practical deployment of the method in process mining tools like ProM 6, with potential for future integration of noise filtering and model conversion.

Proposed method

• Construct log skeletons from activity logs using a set of structural relations: always-before, always-after, and equivalence relations, with extensions beyond standard Declare constraints.
• Use a rule-based classification mechanism where a trace is classified as positive (conforming) if its skeleton is subsumed by the log skeleton, including filtered variants of the log and trace.
• Apply trace filtering during classification: a trace conforms if it conforms to the log skeleton even after selective removal of activities from the log or trace.
• Automatically deduce constraints (e.g., precedence, succession, co-existence via equivalence) from the event log using frequency and co-occurrence analysis.
• Implement the method in ProM 6 as an event log visualizer, enabling interactive inspection of discovered log skeletons.
• Use manual preprocessors (e.g., activity filtering, splitting) in the contest to achieve perfect classification, with future work aiming to automate this preprocessing.

Experimental results

Research questions

• RQ1 Can a classification-based approach to process discovery outperform traditional discovery algorithms in terms of trace classification accuracy?
• RQ2 How effective is a rule-based log skeleton model—based on structural constraints like always-before, always-after, and equivalence—in classifying traces from real-world event logs?
• RQ3 To what extent does the inclusion of trace filtering during classification improve the robustness and accuracy of the model?
• RQ4 Why does the log skeleton approach outperform procedural models like Petri nets and BPMN in classification tasks, especially in the context of the Process Discovery Contest?
• RQ5 Can the log skeleton model be extended to detect and handle noise in event logs, and how would such enhancements affect classification performance?

Key findings

• The log skeleton-based classification method achieved 194 out of 200 correct trace classifications on the 2017 Process Discovery Contest, significantly outperforming the best existing discovery algorithm.
• The Inductive Miner, the state-of-the-art fully-automated discovery algorithm, scored only 147 correct classifications on the same test set.
• The method’s success is attributed to its ability to model complex structural constraints—including non-Declare relations—while preserving classification accuracy under trace filtering.
• Manual preprocessors (e.g., activity filtering and splitting) enabled perfect classification (200/200) in the contest, suggesting that automation of such steps could further improve the fully-automated variant.
• The log skeleton model is sensitive to noise, as missing or spurious activities can break key relations like always-before or equivalence, though the method remains robust when only the end of traces is missing.
• The results suggest a potential bias in the PDC toward procedural models, implying that log skeletons may be better suited for logs designed to challenge such models, offering a new benchmark for future contests.