[Paper Review] Explaining Image Classifiers by Counterfactual Generation

Chun‐Hao Chang, Elliot Creager+2 more|arXiv (Cornell University)|Jul 20, 2018

Generative Adversarial Networks and Image SynthesisComputer Science89 citations

TL;DR

FIDO explains image classifiers by marginalizing over plausible counterfactual inputs using a strong conditional generative model to fill in masked regions, producing concise, distribution-consistent saliency maps.

ABSTRACT

When an image classifier makes a prediction, which parts of the image are relevant and why? We can rephrase this question to ask: which parts of the image, if they were not seen by the classifier, would most change its decision? Producing an answer requires marginalizing over images that could have been seen but weren't. We can sample plausible image in-fills by conditioning a generative model on the rest of the image. We then optimize to find the image regions that most change the classifier's decision after in-fill. Our approach contrasts with ad-hoc in-filling approaches, such as blurring or injecting noise, which generate inputs far from the data distribution, and ignore informative relationships between different parts of the image. Our method produces more compact and relevant saliency maps, with fewer artifacts compared to previous methods.

Motivation & Objective

• Motivate interpretable explanations for differentiable image classifiers by asking which image regions, if unseen, would most change the decision.
• Move beyond heuristic in-filling (e.g., blurring) to conditioning on the data distribution via a generative model.
• Develop a scalable, model-agnostic saliency framework that marginalizes over unobserved pixels.
• Promote compact, artifact-free explanations that respect contextual relationships in images.

Proposed method

• Define saliency as marginalizing over masked image regions using a conditional generative model to sample plausible fills.
• Represent masks with a Bernoulli dropout distribution and optimize its parameters to minimize/maximize classifier confidence under SDR/SSR objectives.
• Use a Concrete/Gumbel-softmax reparameterization to enable gradient-based optimization over binary masks.
• Fill in dropped-out regions with samples from a strong generative model (e.g., CA-GAN) conditioned on the unmasked context.
• Compare SDR and SSR objectives and select SSR for better consistency with training data to reduce artifacts.
• Provide practical algorithms (BBMP vs. FIDO) and discuss implementation details for efficient optimization.

Experimental results

Research questions

• RQ1 How can we identify image regions whose removal or replacement most affects the classifier output while staying within the data distribution?
• RQ2 Does using a strong conditional generative model for in-filling improve the quality and plausibility of saliency maps compared to heuristic filling?
• RQ3 Which objective (SDR or SSR) yields more faithful and less artifact-prone explanations under realistic in-filling?
• RQ4 How do different infilling methods and classifier architectures influence saliency quality and computational efficiency?
• RQ5 Can the approach scale to ImageNet and provide competitive weakly supervised localization performance?

Key findings

• FIDO produces more compact and artifact-free saliency maps than heuristic in-filling or prior methods.
• Generative in-filling (e.g., CA-GAN) yields higher target probabilities and fewer artifacts than heuristic fills across evaluated images.
• Using SSR with strong generative fills provides more consistent explanations than SDR, which can introduce artifacts.
• FIDO-CA saliency maps concentrate on contextual pixels that preserve classifier confidence, often aligning with human-relevant regions while respecting the data manifold.
• Quantitative evaluations on ImageNet show improved localization metrics compared to baselines and prior saliency methods.
• Ablation studies indicate that both the discrete mask modeling (Bernoulli dropout) and expressive in-filling are essential for optimal explanations.