[Paper Review] Conformance checking: A state-of-the-art literature review

Sebastian Dunzer, Matthias Stierle+2 more|arXiv (Cornell University)|Jul 21, 2020

Business Process Modeling and Analysis52 references19 citations

TL;DR

This paper presents a systematic literature review of 37 conformance checking techniques in process mining, classifying them across four dimensions: modelling language, perspective, algorithm type, and quality metric. It reveals a significant research gap in conformance checking for declarative modelling languages and a strong focus on control-flow analysis, calling for future work on multi-perspective techniques including time, roles, and business rules.

ABSTRACT

Conformance checking is a set of process mining functions that compare process instances with a given process model. It identifies deviations between the process instances' actual behaviour ("as-is") and its modelled behaviour ("to-be"). Especially in the context of analyzing compliance in organizations, it is currently gaining momentum -- e.g. for auditors. Researchers have proposed a variety of conformance checking techniques that are geared towards certain process model notations or specific applications such as process model evaluation. This article reviews a set of conformance checking techniques described in 37 scholarly publications. It classifies the techniques along the dimensions "modelling language", "algorithm type", "quality metric", and "perspective" using a concept matrix so that the techniques can be better accessed by practitioners and researchers. The matrix highlights the dimensions where extant research concentrates and where blind spots exist. For instance, process miners use declarative process modelling languages often, but applications in conformance checking are rare. Likewise, process mining can investigate process roles or process metrics such as duration, but conformance checking techniques narrow on analyzing control-flow. Future research may construct techniques that support these neglected approaches to conformance checking.

Motivation & Objective

• To identify current research streams and blind spots in conformance checking within the Information Systems discipline.
• To classify existing conformance checking techniques based on key dimensions: modelling language, perspective, algorithm type, and quality metric.
• To highlight under-researched areas such as declarative models and non-control-flow perspectives (e.g., time, roles, duration).
• To guide future research by identifying opportunities in trace alignment optimization, real-life validation, and integration with rule-based systems.
• To support practitioners and researchers in selecting appropriate conformance checking techniques by providing a structured concept matrix.

Proposed method

• Conducted a systematic literature review using Scopus as the primary database, applying predefined inclusion and exclusion criteria.
• Selected 37 scholarly publications that propose distinct conformance checking techniques using process models and event logs.
• Classified each technique along four dimensions: (1) modelling language (e.g., BPMN, Petri nets, Declare), (2) perspective (control-flow vs. multi-perspective), (3) algorithm type (e.g., log-replay, trace alignments, rule-based, artificial negative events), and (4) quality metric (fitness, precision, simplicity, generalization).
• Constructed a concept matrix to visualize the distribution of techniques across the four dimensions and identify research concentration and blind spots.
• Evaluated bibliometric data using Scopus to assess citation impact and ensure coverage of influential works.
• Excluded articles not strictly focused on conformance checking in process mining, particularly those centered on compliance checking or fraud detection without process model alignment.

Experimental results

Research questions

• RQ1 What are the dominant streams of conformance checking research in the Information Systems discipline?
• RQ2 Which process modelling languages are most commonly supported by existing conformance checking techniques?
• RQ3 To what extent do current techniques consider perspectives beyond control-flow, such as time, roles, or resource usage?
• RQ4 What algorithm types are most prevalent, and how do they differ in implementation and optimization?
• RQ5 What quality metrics are used to evaluate conformance, and are there gaps in the measurement of precision, simplicity, or generalization?

Key findings

• There is a significant lack of conformance checking techniques for declarative process modelling languages such as Declare, despite their growing use in process mining.
• The majority of conformance checking techniques focus exclusively on control-flow analysis, with limited support for multi-perspective analysis involving time, roles, or resource allocation.
• Trace alignments are the most widely used algorithm type, followed by log-replay; rule-based and artificial negative event approaches are classified as 'others'.
• Fitness is the most frequently used quality metric, while precision, simplicity, and generalization are underexplored in current research.
• Only a few studies validate conformance checking techniques using real-life event logs, indicating a gap in empirical evaluation and practical feasibility assessment.
• Despite the growing number of publications, few studies apply conformance checking in real organizational settings, especially outside healthcare, limiting practical insights into scalability and usability.