[Paper Review] Beyond Sparsity: Tree Regularization of Deep Models for Interpretability

Mike Wu, Michael C. Hughes+4 more|arXiv (Cornell University)|Nov 16, 2017

Explainable Artificial Intelligence (XAI)102 citations

TL;DR

This paper introduces tree regularization to train deep models whose predictions are highly accurate yet well-approximated by compact decision trees, improving human-simulability without sacrificing performance across time-series and real-world tasks.

ABSTRACT

The lack of interpretability remains a key barrier to the adoption of deep models in many applications. In this work, we explicitly regularize deep models so human users might step through the process behind their predictions in little time. Specifically, we train deep time-series models so their class-probability predictions have high accuracy while being closely modeled by decision trees with few nodes. Using intuitive toy examples as well as medical tasks for treating sepsis and HIV, we demonstrate that this new tree regularization yields models that are easier for humans to simulate than simpler L1 or L2 penalties without sacrificing predictive power.

Motivation & Objective

• Motivate and define human-simulability as a form of interpretability for deep models.
• Propose tree regularization to encourage decision boundaries that are well-approximated by small decision trees.
• Demonstrate that tree-regularized models achieve high accuracy at low complexity across time-series and real-world domains.
• Provide interpretable tree proxies that mimic deep model predictions to enable clinician and domain expert audit.

Proposed method

• Define an average path length of a decision tree that reproduces the network's thresholded predictions as a complexity measure for simulatability (Omega(W)).
• Train a binary decision tree to mimic the deep model's predictions on a reference dataset, then compute the average path length to measure complexity.
• Develop a differentiable surrogate model hat_Omega(W) using a small MLP to approximate the non-differentiable Omega(W) for gradient-based optimization.
• Train deep models with the surrogate tree-regularization term hat_Omega(W) integrated into the loss (alongside standard loss).
• Optionally construct GRU-HMM hybrids where a tree-regularized GRU residuals the interpretable HMM to improve performance while retaining interpretability.
• Demonstrate the approach with MLPs and GRU-based time-series across synthetic, sepsis, HIV, and TIMIT tasks, with a fidelity analysis showing how well the proxy trees reflect deep-model decisions.

Experimental results

Research questions

• RQ1 Can deep models be trained to have decision boundaries that are easily simulatable by small decision trees without sacrificing predictive accuracy?
• RQ2 Does tree-regularization yield higher accuracy at low tree complexity than L1/L2 penalties across time-series and real-world domains?
• RQ3 Do human-interpretable tree proxies faithfully reflect the predictions of tree-regularized deep models (fidelity)?
• RQ4 Can hybrid models (GRU-HMM) leverage tree-regularized deep components to improve performance while maintaining interpretability?
• RQ5 Are the resulting tree proxies clinically or auditorily meaningful for domain experts?

Key findings

• Tree-regularized models achieve higher AUC at small average path lengths than L1/L2-regularized counterparts across tasks.
• On the signal-and-noise HMM task, tree-regularized GRUs reach AUC near 0.9 with path length around 10, while L1/L2 require longer paths for similar AUC.
• On Sepsis, GRU models with tree-regularization show AUC gains of 0.05–0.1 at path lengths of 2–10.
• On TIMIT and HIV tasks, tree-regularization yields AUC gains of 0.05–0.15 at moderate path lengths, while standalone trees or L1/L2 struggle at low complexity.
• Decision-tree proxies faithfully reflect the deep model predictions with fidelity scores in the range 0.81–0.93 across tasks.
• Decision-tree proxies are interpretable (path length ≤ 25) and helpful for clinical reasoning and trust.
• GRU-HMM hybrids with tree-regularized GRUs achieve higher accuracy at lower path lengths than the GRU alone for equivalent complexity.