[Paper Review] Counterfactual Visual Explanations

Yash Goyal, Ziyan Wu+4 more|arXiv (Cornell University)|Apr 16, 2019

Explainable Artificial Intelligence (XAI)31 references38 citations

TL;DR

The paper presents a method to generate counterfactual visual explanations by editing regions of a query image with regions from a distractor image to change a model's prediction, and demonstrates its interpretability and teaching usefulness across multiple datasets.

ABSTRACT

In this work, we develop a technique to produce counterfactual visual explanations. Given a 'query' image $I$ for which a vision system predicts class $c$, a counterfactual visual explanation identifies how $I$ could change such that the system would output a different specified class $c'$. To do this, we select a 'distractor' image $I'$ that the system predicts as class $c'$ and identify spatial regions in $I$ and $I'$ such that replacing the identified region in $I$ with the identified region in $I'$ would push the system towards classifying $I$ as $c'$. We apply our approach to multiple image classification datasets generating qualitative results showcasing the interpretability and discriminativeness of our counterfactual explanations. To explore the effectiveness of our explanations in teaching humans, we present machine teaching experiments for the task of fine-grained bird classification. We find that users trained to distinguish bird species fare better when given access to counterfactual explanations in addition to training examples.

Motivation & Objective

• Motivate and formalize counterfactual visual explanations for deep vision models.
• Define a minimal-edit transformation that replaces image regions to change the model’s output to a specified distractor class.
• Show interpretability and discriminativeness of counterfactual explanations across datasets.
• Demonstrate the potential of counterfactual explanations for machine teaching of fine-grained tasks.

Proposed method

• Decompose a CNN into a spatial feature extractor f(I) and a decision network g(f(I)).
• Define a transformation f(I) -> f(I*) = (1-a) ∘ f(I) + a ∘ (P f(I′)) using a binary mask a and a permutation P to align regions.
• Formulate the minimum-edit counterfactual problem: minimize ||a||1 subject to c′ = argmax g(f(I*)).
• Solve via greedy sequential exhaustive search that selects edits maximizing the distractor class log-probability g_c′, iterating until the decision changes.
• Provide a continuous relaxation by relaxing a to a simplex and P to a right stochastic matrix, with softmax reparameterizations a = σ(α) and P_i· = σ(m_i^T).
• Evaluate on SHAPES, MNIST, Omniglot, and CUB with qualitative results and quantitative metrics such as edit counts and runtimes.

Experimental results

Research questions

• RQ1 How can we generate faithful counterfactual visual explanations that reveal what needs to change for a model to predict a different class?
• RQ2 Do counterfactual visual explanations help humans learn and discriminate fine-grained categories (machine teaching) beyond standard training examples?
• RQ3 How do explanations align with discriminative image regions and keypoints across datasets?
• RQ4 How stable are explanations with different distractor choices (random vs nearest-neighbor)?

Key findings

• Counterfactual explanations identify discriminative region edits enabling a target distractor class, with interpretable region pairings across datasets.
• MNIST results show an average of 2.67 edits to flip to a distractor class, with 15 μs per image on Titan Xp.
• Omniglot results show an average of 1.46 edits per image with 9 μs per image runtime.
• CUB experiments report average edits of 7.4 (random distractor) and 5.3 (nearest-neighbor distractor) with runtimes of 1.85 s and 1.34 s per image, respectively.
• Machine teaching on CUB shows mean test accuracy of 78.77% with counterfactual explanations, 74.29% with GradCAM, and 71.09% with no explanations; familiarity boosts performance to 72.4% with counterfactuals vs 61.7% without explanations.
• Discriminative regions often lie inside bird segmentations (97%) and align with keypoints 75-80% of the time; explanations can highlight correct class attributes even when predictions are wrong.