[Paper Review] Neural Network Attributions: A Causal Perspective

Aditya Chattopadhyay, Piyushi Manupriya+2 more|arXiv (Cornell University)|Feb 6, 2019

Explainable Artificial Intelligence (XAI)Computer Science34 references32 citations

TL;DR

The paper introduces a causal attribution method for neural networks by modeling networks as Structural Causal Models and computing Average Causal Effects of input neurons on outputs, scalable to high dimensions and applicable to RNNs.

ABSTRACT

We propose a new attribution method for neural networks developed using first principles of causality (to the best of our knowledge, the first such). The neural network architecture is viewed as a Structural Causal Model, and a methodology to compute the causal effect of each feature on the output is presented. With reasonable assumptions on the causal structure of the input data, we propose algorithms to efficiently compute the causal effects, as well as scale the approach to data with large dimensionality. We also show how this method can be used for recurrent neural networks. We report experimental results on both simulated and real datasets showcasing the promise and usefulness of the proposed algorithm.

Motivation & Objective

• Motivate interpretable, causally grounded explanations for neural network predictions.
• Model neural networks as Structural Causal Models (SCMs) to define and compute causal attributions.
• Develop scalable algorithms to estimate interventional expectations and ACE in high-dimensional settings.
• Extend the approach to recurrent architectures and address practical computation concerns.

Proposed method

• Represent feedforward networks as SCMs and marginalize hidden layers to obtain a reduced causal model.
• Define Average Causal Effect (ACE) of an input neuron on an output neuron using do-operations.
• Compute interventional expectations E[y|do(x_i=α)] via a Taylor expansion around interventional means and interventional covariances.
• Introduce causal regressors (polynomial functions learned via Bayesian linear regression) to estimate interventional expectations efficiently.
• Provide Phase I (compute interventional expectations) and Phase II (learn causal regressors and baseline) workflows.
• Discuss extension to RNNs with time-unfolded SCMs and handling of dependencies across timesteps.

Experimental results

Research questions

• RQ1 What is the causal effect of a specific input neuron on a specific output neuron in a neural network?
• RQ2 How can we compute and approximate interventional expectations and ACE in high-dimensional neural networks?
• RQ3 Can this causal attribution framework be extended to recurrent architectures?
• RQ4 How do causal attributions compare to gradient-based or perturbation-based methods in terms of biases and robustness?

Key findings

• Proposes a principled causal attribution method based on SCMs and ACE that marginalizes other inputs to avoid bias from inter-feature correlations.
• Shows how to compute interventional expectations using a Taylor expansion and interventional covariances, enabling scalable ACE estimation.
• Introduces causal regressors to estimate ACE on-the-fly, reducing computational cost.
• Extends the approach to Recurrent Neural Networks with appropriate marginalization considerations.
• Empirical results on Iris, simulated data, and airplane trajectory data illustrate the usefulness and advantages over gradient-based baselines in certain settings.