[Paper Review] Explaining an image classifier's decisions using generative models

Chirag Agarwal, Anh‐Tu Nguyen|arXiv (Cornell University)|Oct 9, 2019

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper proposes integrating a generative inpainter into attribution methods to generate more realistic counterfactual images when explaining classifier decisions. By replacing heuristic perturbations (e.g., blurring or masking) with realistic image generation, the method improves attribution quality across object localization, deletion, and saliency metrics on ImageNet and Places365.

ABSTRACT

Perturbation-based explanation methods often measure the contribution of an input feature to an image classifier's outputs by heuristically removing it via e.g. blurring, adding noise, or graying out, which often produce unrealistic, out-of-samples. Instead, we propose to integrate a generative inpainter into three representative attribution methods to remove an input feature. Our proposed change improved all three methods in (1) generating more plausible counterfactual samples under the true data distribution; (2) being more accurate according to three metrics: object localization, deletion, and saliency metrics; and (3) being more robust to hyperparameter changes. Our findings were consistent across both ImageNet and Places365 datasets and two different pairs of classifiers and inpainters.

Motivation & Objective

• To address the limitation of perturbation-based explanation methods that produce unrealistic, out-of-distribution counterfactual samples.
• To improve the fidelity and plausibility of counterfactual images used in explaining image classifier decisions.
• To enhance the accuracy and robustness of attribution methods using realistic image generation instead of heuristic feature removal.
• To evaluate the impact of generative inpainting on multiple attribution methods across diverse datasets and model pairs.

Proposed method

• Integrates a pre-trained generative inpainter into three representative attribution methods to replace heuristic feature removal with realistic image reconstruction.
• Uses the inpainter to generate counterfactual images by masking specific input features and filling them with plausible content based on context.
• Applies the modified attribution methods to compute feature importance scores using the generated counterfactuals under the true data distribution.
• Employs standard evaluation metrics—object localization, deletion, and saliency—to assess attribution quality.
• Trains and evaluates the approach on ImageNet and Places365 using two distinct classifier-inpainter pairs to ensure generalization.
• Validates that the inpainting-based method produces more realistic counterfactuals compared to baseline perturbation techniques.

Experimental results

Research questions

• RQ1 Does replacing heuristic perturbations with generative inpainting improve the realism of counterfactual images in attribution methods?
• RQ2 How does the use of generative inpainting affect the accuracy of attribution methods across standard evaluation metrics?
• RQ3 To what extent does the proposed method improve robustness to hyperparameter changes compared to baseline methods?
• RQ4 Does the performance gain hold across different datasets and classifier-inpainter combinations?
• RQ5 Can generative inpainting produce more faithful explanations by better reflecting the true data distribution?

Key findings

• The proposed method generates counterfactual samples that are significantly more plausible and realistic than those produced by heuristic perturbation methods.
• All three attribution methods showed improved performance on object localization, deletion, and saliency metrics when using generative inpainting.
• The method demonstrated greater robustness to hyperparameter variations compared to baseline approaches.
• The improvements were consistent across both ImageNet and Places365 datasets.
• The gains were observed across two different pairs of classifiers and inpainters, indicating generalizability of the approach.
• The results confirm that using realistic counterfactuals under the true data distribution leads to more reliable and accurate model explanations.