[Paper Review] A Bayesian Approach to Rule Mining

Luis I. Lopera González, Adrian Derungs+1 more|arXiv (Cornell University)|Dec 13, 2019

Data Mining Algorithms and Applications23 references4 citations

TL;DR

This paper proposes Bayesian Rule Mining (BRM), a novel approach to association rule mining that uses a recursive Bayesian update to compute rule belief, ensuring belief increases with each new observation. Unlike frequent association rule mining (FRM), BRM avoids support dilution and effectively extracts rare, meaningful rules by focusing on individual process dynamics rather than commonalities across datasets.

ABSTRACT

In this paper, we introduce the increasing belief criterion in association rule mining. The criterion uses a recursive application of Bayes' theorem to compute a rule's belief. Extracted rules are required to have their belief increase with their last observation. We extend the taxonomy of association rule mining algorithms with a new branch for Bayesian rule mining~(BRM), which uses increasing belief as the rule selection criterion. In contrast, the well-established frequent association rule mining~(FRM) branch relies on the minimum-support concept to extract rules. We derive properties of the increasing belief criterion, such as the increasing belief boundary, no-prior-worries, and conjunctive premises. Subsequently, we implement a BRM algorithm using the increasing belief criterion, and illustrate its functionality in three experiments: (1)~a proof-of-concept to illustrate BRM properties, (2)~an analysis relating socioeconomic information and chemical exposure data, and (3)~mining behaviour routines in patients undergoing neurological rehabilitation. We illustrate how BRM is capable of extracting rare rules and does not suffer from support dilution. Furthermore, we show that BRM focuses on the individual event generating processes, while FRM focuses on their commonalities. We consider BRM's increasing belief as an alternative criterion to thresholds on rule support, as often applied in FRM, to determine rule usefulness.

Motivation & Objective

• To address support dilution in frequent association rule mining (FRM), where rule support decreases as dataset size grows.
• To overcome FRM's limitation in extracting rare but consistent rules due to minimum-support thresholds.
• To propose a new branch in association rule mining taxonomy—Bayesian Rule Mining (BRM)—using belief as a rule selection criterion.
• To demonstrate BRM’s ability to extract rules that reveal individual process dynamics, especially in heterogeneous datasets.
• To show BRM’s viability in real-world applications such as socioeconomic data and neurological rehabilitation.

Proposed method

• Proposes the increasing belief criterion, which recursively applies Bayes’ theorem to update rule belief with each new observation.
• Derives key properties: increasing belief boundary, no-prior-worries, and conjunctive premises, ensuring rule stability and interpretability.
• Implements an exhaustive search BRM algorithm using belief updates based on prior probability, observed data, and likelihood.
• Uses a filtering stage post-mining to remove irrelevant rules, with filter choice dependent on application context (e.g., confidence threshold or best confidence per conclusion).
• Applies the belief update formula: belief = p * P(data|rule) / P(data), recursively updated as new data arrives.
• Validates BRM using three datasets: synthetic time series, socioeconomic-chemical exposure data, and patient behavior routines in rehabilitation.

Experimental results

Research questions

• RQ1 Can a belief-based criterion replace minimum-support thresholds in rule mining to avoid support dilution?
• RQ2 Can BRM effectively extract rare but consistent rules that FRM misses due to low support?
• RQ3 Does BRM reveal individual process dynamics better than FRM, which emphasizes commonalities?
• RQ4 How does BRM perform in real-world datasets with heterogeneous processes and rare events?
• RQ5 Can BRM be used to group patients into behavioral subtypes, such as active vs. sedentary, based on routine mining?

Key findings

• BRM successfully extracts rare rules without suffering from support dilution, even as dataset size increases.
• In the socioeconomic data experiment, BRM identified rare but meaningful associations, such as low poverty score correlating with high education levels.
• In the rehabilitation study, BRM grouped patients into active and sedentary behavioral subtypes, while FRM failed to detect such groupings.
• The PEP analysis showed that FRM rules remained unchanged across patient subsets, indicating they reflect population-wide commonalities, whereas BRM rules changed, revealing process-specific patterns.
• BRM’s belief metric is robust to dataset size and can be tuned using process knowledge, such as expected frequency ratios and transition times.
• Filtering strategies must be tailored to the application: confidence thresholds work well for separating processes, while best-confidence-per-conclusion filters suit transition mining.