[Paper Review] Gaining Free or Low-Cost Transparency with Interpretable Partial Substitute

Tong Wang|arXiv (Cornell University)|Feb 12, 2018

Machine Learning and Data Classification33 references5 citations

TL;DR

This paper proposes a hybrid model framework that replaces a black-box model with an interpretable partial substitute on data subsets where the black-box is overkill, achieving transparency at no or minimal cost to predictive performance. The Hybrid Rule Sets (HyRS) model uses interpretable decision rules for early, transparent predictions on a subset of data, while deferring to the black-box only when necessary, demonstrating a trade-off between transparency and accuracy with 82% transparency and 77% accuracy in experiments.

ABSTRACT

This work addresses the situation where a black-box model with good predictive performance is chosen over its interpretable competitors, and we show interpretability is still achievable in this case. Our solution is to find an interpretable substitute on a subset of data where the black-box model is overkill or nearly overkill while leaving the rest to the black-box. This transparency is obtained at minimal cost or no cost of the predictive performance. Under this framework, we develop a Hybrid Rule Sets (HyRS) model that uses decision rules to capture the subspace of data where the rules are as accurate or almost as accurate as the black-box provided. To train a HyRS, we devise an efficient search algorithm that iteratively finds the optimal model and exploits theoretically grounded strategies to reduce computation. Our framework is agnostic to the black-box during training. Experiments on structured and text data show that HyRS obtains an effective trade-off between transparency and interpretability.

Motivation & Objective

• To address the challenge of achieving model interpretability when high-performing black-box models are preferred over interpretable alternatives.
• To develop a method that provides full-fidelity interpretability on a subset of data where the black-box model is overkill, avoiding post-hoc approximation errors.
• To minimize the performance cost of interpretability by restricting the interpretable model to a data subspace where it performs nearly as well as the black-box.
• To enable early, cost-effective predictions using only time-invariant features by leveraging interpretable rules for a portion of the data.
• To provide a generalizable framework that integrates any interpretable model with a black-box, supporting diverse model types like linear models, decision trees, and prototype-based models.

Proposed method

• The framework uses a sequential decision process: an interpretable model (e.g., rule sets) is applied first; if it is confident, it predicts; otherwise, the black-box model is invoked.
• Hybrid Rule Sets (HyRS) are constructed using association rules as interpretable local substitutes, trained on data where the black-box model is nearly redundant.
• An efficient search algorithm is developed to iteratively identify optimal rule sets, leveraging theoretical strategies to reduce computational cost.
• The method is agnostic to the black-box model during training, allowing integration with any existing black-box model.
• The model is trained using only time-invariant features available at case launch, enabling early prediction without waiting for time-series data.
• A trade-off curve between transparency (proportion of data covered by rules) and accuracy is generated by varying a threshold parameter (θ₂), enabling users to select optimal model configurations.

Experimental results

Research questions

• RQ1 Can we achieve full-fidelity interpretability on a subset of data where a black-box model is overkill, without sacrificing predictive performance?
• RQ2 What is the trade-off between model transparency (proportion of data explained by interpretable rules) and predictive accuracy in a hybrid model setting?
• RQ3 Can interpretable models be used effectively as partial substitutes to reduce reliance on black-box models while maintaining high performance?
• RQ4 How can we efficiently train interpretable models to cover a maximal, high-accuracy subspace of data without prior knowledge of the black-box's decision boundaries?
• RQ5 Can the proposed framework be generalized to support various interpretable models (e.g., linear models, decision trees) alongside any black-box model?

Key findings

• The HyRS model achieves 82% transparency (82% of data covered by interpretable rules) with 77% predictive accuracy, outperforming purely interpretable models in accuracy while maintaining interpretability on a significant data subset.
• The model covers 82% of the data with only four positive and three negative rules, meaning 18% of instances are still routed to the black-box model, minimizing reliance on complex models.
• The framework enables a trade-off between transparency and accuracy, with the curve in Figure 4 showing that increasing transparency leads to a gradual, predictable decline in accuracy, but always above the performance of pure interpretable models.
• The use of interpretable rules on time-invariant features alone allows for early prediction in medical crowdfunding, reducing dependency on costly or time-consuming data collection.
• The method achieves high predictive performance (test accuracy of 0.96) with a deep LSTM model, which is then used as the black-box reference, validating the framework's effectiveness in real-world settings.
• The framework is generalizable and supports integration with diverse interpretable models, including linear models, decision trees, and prototype-based models, as demonstrated in the paper’s broader context.