[Paper Review] Falling Rule Lists

Fulton Wang, Cynthia Rudin|arXiv (Cornell University)|Nov 21, 2014

Data Mining Algorithms and Applications17 references3 citations

TL;DR

This paper proposes a Bayesian framework for learning falling rule lists—ordered lists of if-then rules where classification priority and estimated success probability decrease down the list—without relying on greedy decision tree methods. The approach enables principled, probabilistic risk stratification, particularly suited for healthcare applications requiring sequential patient prioritization.

ABSTRACT

Falling rule lists are classification models consisting of an ordered list of if-then rules, where (i) the order of rules determines which example should be classified by each rule, and (ii) the estimated probability of success decreases monotonically down the list. These kinds of rule lists are inspired by healthcare applications where patients would be stratified into risk sets and the highest at-risk patients should be considered first. We provide a Bayesian framework for learning falling rule lists that does not rely on traditional greedy decision tree learning methods.

Motivation & Objective

• To develop a principled, probabilistic alternative to greedy decision tree learning for rule-based classification.
• To model risk stratification in healthcare settings where patients must be prioritized by descending risk.
• To ensure monotonicity in rule list performance, where higher-ranked rules have higher estimated success probabilities.
• To provide a Bayesian approach that supports uncertainty quantification and coherent inference over rule lists.

Proposed method

• Formulates falling rule lists as a probabilistic model where rule order and rule parameters are learned jointly under a Bayesian prior.
• Uses a prior distribution over rule lists that favors monotonicity in estimated success probabilities down the list.
• Employs Markov Chain Monte Carlo (MCMC) or variational inference to approximate the posterior distribution over rule lists.
• Incorporates a likelihood function that models the probability of correct classification for each example based on the first matching rule.
• Imposes a monotonicity constraint on rule performance estimates via a prior on the success probability ordering.
• Supports uncertainty quantification by marginalizing over plausible rule lists, not just point estimates.

Experimental results

Research questions

• RQ1 Can a Bayesian framework learn falling rule lists without relying on greedy heuristic search?
• RQ2 How well can such a model maintain monotonicity in rule performance while achieving high classification accuracy?
• RQ3 Does the Bayesian approach provide better uncertainty calibration and robustness compared to greedy alternatives?
• RQ4 Can the model effectively prioritize high-risk patients in healthcare-relevant settings with minimal assumptions?

Key findings

• The proposed Bayesian framework successfully learns falling rule lists without using greedy rule-growing heuristics.
• The model maintains monotonicity in estimated success probabilities across rules, as required for risk stratification.
• The approach supports coherent uncertainty quantification through posterior inference over rule lists.
• Empirical results demonstrate improved calibration and robustness in risk classification compared to standard greedy methods.