[Paper Review] Automatic Relevance Determination Bayesian Neural Networks for Credit Card Default Modelling

Rendani Mbuvha, Ilyes Boulkaibet+1 more|arXiv (Cornell University)|Jun 14, 2019

Financial Distress and Bankruptcy Prediction4 citations

TL;DR

This paper proposes Bayesian Neural Networks (BNNs) with Automatic Relevance Determination (ARD) for credit card default prediction, using both Gaussian approximation and Hybrid Monte Carlo (HMC) inference. Results show ARD-BNNs significantly outperform standard BNNs, with HMC-ARD achieving slightly better false negative control and both methods providing robust, interpretable feature importance rankings critical for regulatory compliance.

ABSTRACT

Credit risk modelling is an integral part of the global financial system. While there has been great attention paid to neural network models for credit default prediction, such models often lack the required interpretation mechanisms and measures of the uncertainty around their predictions. This work develops and compares Bayesian Neural Networks(BNNs) for credit card default modelling. This includes a BNNs trained by Gaussian approximation and the first implementation of BNNs trained by Hybrid Monte Carlo(HMC) in credit risk modelling. The results on the Taiwan Credit Dataset show that BNNs with Automatic Relevance Determination(ARD) outperform normal BNNs without ARD. The results also show that BNNs trained by Gaussian approximation display similar predictive performance to those trained by the HMC. The results further show that BNN with ARD can be used to draw inferences about the relative importance of different features thus critically aiding decision makers in explaining model output to consumers. The robustness of this result is reinforced by high levels of congruence between the features identified as important using the two different approaches for training BNNs.

Motivation & Objective

• To address the lack of interpretability and uncertainty quantification in traditional neural networks for credit risk modeling.
• To develop Bayesian Neural Networks (BNNs) that enable feature importance estimation via Automatic Relevance Determination (ARD).
• To compare two Bayesian inference methods—Gaussian approximation and Hybrid Monte Carlo (HMC)—in the context of credit default prediction.
• To evaluate whether ARD-BNNs can enhance model transparency and support regulatory compliance, such as the GDPR's 'right to explanation'.
• To assess the robustness and predictive performance of ARD-BNNs using real-world credit data.

Proposed method

• Employs Bayesian Neural Networks (BNNs) to model credit default probability with uncertainty quantification.
• Applies Automatic Relevance Determination (ARD) to infer the relative importance of input features by learning inverse variance parameters of weight priors.
• Uses Gaussian approximation for approximate Bayesian inference in BNNs, enabling fast training and uncertainty estimation.
• Implements Hybrid Monte Carlo (HMC) for exact posterior sampling, improving theoretical robustness of uncertainty and feature relevance estimates.
• Trains and evaluates models on the Taiwan Credit Card Dataset using a 70:30 train-test split.
• Evaluates performance using AUC and confusion matrices, with focus on false negative rates relevant to credit risk.

Experimental results

Research questions

• RQ1 Can Bayesian Neural Networks with Automatic Relevance Determination (ARD) improve interpretability in credit default modeling compared to standard neural networks?
• RQ2 How do Gaussian approximation and Hybrid Monte Carlo (HMC) inference compare in terms of predictive performance and uncertainty estimation in ARD-BNNs for credit risk?
• RQ3 Do ARD-BNNs identify consistent and robust feature importance rankings across different inference methods?
• RQ4 Does the inclusion of ARD in BNNs lead to better generalization and reduced false negative rates in credit default prediction?
• RQ5 Can HMC-ARD provide more reliable uncertainty estimates and feature relevance than Gaussian approximation in real-world credit data?

Key findings

• ARD-BNNs trained with HMC achieved an AUC of 0.7783, outperforming the HMC-MLP baseline (AUC 0.7079), demonstrating the benefit of ARD for model accuracy.
• The Gaussian approximation-ARD model achieved an AUC of 0.7753, showing comparable predictive performance to HMC-ARD despite being computationally less intensive.
• HMC-ARD showed a marginally lower number of false negatives (331) compared to Gaussian approximation-ARD (355), indicating better control over costly default misclassifications.
• Both ARD-BNNs identified PAY0 (payment status six months ago) as the most important feature, with strong agreement on other key predictors like PAY3, LIMIT BAL, and PAY AMT2.
• The HMC-ARD model additionally identified education level as a relevant predictor, and the posterior variance heatmaps confirmed consistent feature importance rankings across both inference methods.
• The high congruence in feature relevance between HMC-ARD and Gaussian approximation-ARD indicates robustness in ARD-based feature selection, supporting its use for model interpretation.