[Paper Review] GAMI-Net: An Explainable Neural Network based on Generalized Additive Models with Structured Interactions

Zebin Yang, Aijun Zhang+1 more|arXiv (Cornell University)|Mar 16, 2020

Explainable Artificial Intelligence (XAI)37 references16 citations

TL;DR

GAMI-Net is an intrinsically explainable neural network that extends generalized additive models by modeling main effects and structured pairwise interactions through disentangled subnetworks. It enforces sparsity, heredity, and marginal clarity constraints to ensure interpretability while achieving competitive prediction accuracy, outperforming or matching models like EBM on real-world datasets with superior model transparency.

ABSTRACT

The lack of interpretability is an inevitable problem when using neural network models in real applications. In this paper, an explainable neural network based on generalized additive models with structured interactions (GAMI-Net) is proposed to pursue a good balance between prediction accuracy and model interpretability. GAMI-Net is a disentangled feedforward network with multiple additive subnetworks; each subnetwork consists of multiple hidden layers and is designed for capturing one main effect or one pairwise interaction. Three interpretability aspects are further considered, including a) sparsity, to select the most significant effects for parsimonious representations; b) heredity, a pairwise interaction could only be included when at least one of its parent main effects exists; and c) marginal clarity, to make main effects and pairwise interactions mutually distinguishable. An adaptive training algorithm is developed, where main effects are first trained and then pairwise interactions are fitted to the residuals. Numerical experiments on both synthetic functions and real-world datasets show that the proposed model enjoys superior interpretability and it maintains competitive prediction accuracy in comparison to the explainable boosting machine and other classic machine learning models.

Motivation & Objective

• To address the lack of interpretability in deep neural networks by developing an intrinsically explainable model.
• To balance predictive accuracy with model transparency using structured interactions and enforceable interpretability constraints.
• To improve upon existing explainable models like EBM by incorporating sparsity, heredity, and marginal clarity in a neural network framework.
• To enable visualization of main effects and pairwise interactions through 1D and 2D plots for practical interpretability.

Proposed method

• GAMI-Net is a disentangled feedforward neural network composed of multiple additive subnetworks, each modeling a single main effect or pairwise interaction.
• Each subnetwork uses a multi-layer perceptron to represent nonlinear shape functions of the input features or their interactions.
• The model enforces sparsity via L1 regularization to select only significant effects, promoting model parsimony.
• Heredity constraint ensures that a pairwise interaction is only included if at least one of its main effects is present, maintaining structural interpretability.
• Marginal clarity is enforced to prevent overlap between main effects and interactions, ensuring distinct and interpretable contributions.
• An adaptive training algorithm first trains main effects and then fits pairwise interactions to the residuals, improving convergence and interpretability.

Experimental results

Research questions

• RQ1 Can a neural network be designed to be intrinsically interpretable while maintaining competitive predictive performance?
• RQ2 How can structured interactions be incorporated into a neural network framework to ensure model transparency?
• RQ3 To what extent do sparsity, heredity, and marginal clarity constraints improve interpretability without sacrificing accuracy?
• RQ4 How does GAMI-Net compare to state-of-the-art explainable models like EBM in terms of predictive performance and interpretability?

Key findings

• GAMI-Net achieves competitive predictive performance across 16 real-world datasets, with mean test RMSE values comparable to or better than EBM and other benchmark models.
• On the California housing dataset, GAMI-Net achieved a test RMSE of 4.708 ± 0.087, outperforming EBM (5.777 ± 0.169) and other models.
• On the electrical grid dataset, GAMI-Net achieved a test RMSE of 0.658 ± 0.019, significantly outperforming EBM (1.448 ± 0.028).
• The model consistently maintained low RMSE across diverse datasets, including abalone, wind, and elevators, demonstrating robust generalization.
• GAMI-Net’s interpretability is enhanced by clear visualization of main effects (1D plots) and pairwise interactions (2D heatmaps), enabling domain experts to validate model behavior.
• The adaptive training procedure, which trains main effects before interactions, led to stable convergence and improved model fidelity.