[Paper Review] An Interpretable Model with Globally Consistent Explanations for Credit Risk

Chaofan Chen, Kangcheng Lin+4 more|arXiv (Cornell University)|Nov 30, 2018

Explainable Artificial Intelligence (XAI)14 references65 citations

TL;DR

The paper presents a globally interpretable two-layer additive risk model for credit default prediction, with monotonicity constraints, subscale interpretability, and SetCoverExplanation for consistent rule-based summaries plus an interactive visualization tool.

ABSTRACT

We propose a possible solution to a public challenge posed by the Fair Isaac Corporation (FICO), which is to provide an explainable model for credit risk assessment. Rather than present a black box model and explain it afterwards, we provide a globally interpretable model that is as accurate as other neural networks. Our "two-layer additive risk model" is decomposable into subscales, where each node in the second layer represents a meaningful subscale, and all of the nonlinearities are transparent. We provide three types of explanations that are simpler than, but consistent with, the global model. One of these explanation methods involves solving a minimum set cover problem to find high-support globally-consistent explanations. We present a new online visualization tool to allow users to explore the global model and its explanations.

Motivation & Objective

• Provide a globally interpretable alternative to black-box credit risk models without sacrificing accuracy.
• Decompose features into meaningful subscales to enable interpretability and visualization.
• Ensure monotonicity constraints on features to reflect domain knowledge in credit risk.
• Develop consistent, rule-based explanations and case-based visuals aligned with the global model.
• Offer an interactive tool to explore the model, explanations, and subscale contributions.

Proposed method

• Propose a two-layer additive risk model (ARM) where features are grouped into subscales and transformed via monotone piecewise-constant functions.
• Impose non-negativity constraints on step-function coefficients to ensure monotonic relationships.
• Compute subscale scores with sigmoid transformations, then combine them linearly and pass through a final sigmoid to yield default probability.
• Train subscales independently with regularization or jointly with the global objective, preserving sparsity and monotonicity.
• Provide three explanation types: (i) variable importance via subscale contributions, (ii) SetCoverExplanation to generate globally-consistent conjunctive rules, (iii) case-based explanations by finding similar historical cases.
• Develop an online visualization tool to display the full model, subscale contributions, and explanations.

Experimental results

Research questions

• RQ1 Can a globally interpretable model match the accuracy of neural networks for credit risk while remaining transparent?
• RQ2 How can feature subscales be designed to be meaningful, sparse, and monotone with respect to risk factors?
• RQ3 Can explanations be made globally consistent using set-cover based rules without sacrificing fidelity?
• RQ4 How effective are interactive visualizations for users to explore subscale contributions and explanations?
• RQ5 What is the trade-off between rule sparsity and coverage in SetCoverExplanation for credit risk data?

Key findings

• The two-layer additive risk model achieves accuracy comparable to other machine learning methods while remaining interpretable.
• Features are partitioned into 10 subscales from 23 original features, each producing a subscale risk score with a meaningful interpretation.
• Monotonicity constraints are enforced by using one-sided interval step functions with non-negative coefficients, ensuring monotone relationships for constrained features.
• SetCoverExplanation generates globally-consistent, high-support rules that summarize model behavior and can describe broad patterns in predictions.
• An interactive visualization tool demonstrates the full computation, subscale contributions, and case-based explanations, aiding trust and understanding.
• The approach yields interpretable global explanations and local, case-based insights without resorting to opaque black-box models.