[Paper Review] The Intriguing Relation Between Counterfactual Explanations and Adversarial Examples

Timo Freiesleben|arXiv (Cornell University)|Sep 11, 2020

Adversarial Robustness in Machine Learning138 references46 citations

TL;DR

This paper formalizes the mathematical relationship between counterfactual explanations (CEs) and adversarial examples (AEs), showing they stem from the same optimization problem but differ in key properties: CEs require proximity to the original input and a flip to a desired, correct label, while AEs require proximity but aim for misclassification. The study clarifies conceptual distinctions, unifies terminology, and identifies methodological synergies between XAI and adversarial robustness research.

ABSTRACT

The same method that creates adversarial examples (AEs) to fool image-classifiers can be used to generate counterfactual explanations (CEs) that explain algorithmic decisions. This observation has led researchers to consider CEs as AEs by another name. We argue that the relationship to the true label and the tolerance with respect to proximity are two properties that formally distinguish CEs and AEs. Based on these arguments, we introduce CEs, AEs, and related concepts mathematically in a common framework. Furthermore, we show connections between current methods for generating CEs and AEs, and estimate that the fields will merge more and more as the number of common use-cases grows.

Motivation & Objective

• . The paper aims to resolve conceptual confusion between counterfactual explanations (CEs) and adversarial examples (AEs), which are often conflated despite differing core properties.
• It seeks to unify the terminology and formalism of CEs and AEs within a single mathematical framework to improve clarity and reduce ambiguity in the literature.
• The study investigates how methods for generating CEs can inform AE generation and vice versa, especially in shared application domains like lending and image classification.
• It highlights the risks of misapplying techniques—such as surrogate models or sensitivity analysis—when generating CEs, due to lack of guaranteed proximity to the original input.
• The paper advocates for a deeper integration of XAI and adversarial machine learning, grounded in formal distinctions to guide future research.

Proposed method

• . The paper introduces a unified mathematical framework based on the optimization problem: argmin_x′∈X d(x, x′) + λ d′(f(x′), y_des), which underlies both CEs and AEs.
• It formally defines CEs as inputs x′ that are close to x (minimizing d(x, x′)) and result in a desired, correct prediction y_des (minimizing d′(f(x′), y_des)).
• AEs are defined as inputs x′ that are close to x but result in misclassification, i.e., f(x′) ≠ y_true, where y_true is the ground-truth label.
• The paper distinguishes CEs from AEs based on two key criteria: (1) the target label (desired vs. incorrect), and (2) the tolerance for proximity (maximal closeness is essential for CEs but not for AEs).
• It evaluates existing methods for generating CEs and AEs, identifying which are conceptually transferable (e.g., gradient-based optimization) and which are not (e.g., surrogate models for CEs in tabular data).
• The study proposes causal modeling as a promising path for formalizing misclassification without relying on ground-truth labels, suggesting that AEs may exploit non-causal, irrelevant feature perturbations.

Experimental results

Research questions

• RQ1 . How do counterfactual explanations and adversarial examples relate mathematically, and in what ways are they formally distinct?
• RQ2 What are the core conceptual differences between CEs and AEs, particularly regarding proximity and target label semantics?
• RQ3 To what extent can methods for generating CEs be adapted to generate AEs, and vice versa, without conceptual or methodological flaws?
• RQ4 Why do current methods for generating CEs often fail to ensure maximal proximity to the original input, and what are the consequences for explanatory power?
• RQ5 Can causal modeling provide a formal basis for distinguishing justified from unjustified misclassifications, thereby improving the conceptual foundation of AE generation?

Key findings

• . The same optimization problem generates both counterfactual explanations and adversarial examples, but the two are formally distinct due to differing objectives: CEs aim for a correct, desired label, while AEs aim for misclassification.
• Maximal proximity to the original input is a defining feature of counterfactual explanations, but this is not guaranteed by many current generation methods such as sensitivity analysis or surrogate models.
• Gradient-based optimization techniques from the AE literature can be conceptually transferred to CE generation, as they naturally enforce proximity.
• Surrogate model approaches for CEs may produce misleading explanations if the surrogate is not faithful to the original model, especially in tabular data settings.
• The paper identifies that current AE generation often relies on poorly justified distance metrics, and that model behavior—particularly failure to learn human-robust concepts—may be a deeper cause of AE vulnerability than distance metrics alone.
• Causal modeling offers a promising, formal path to define misclassification as a violation of causal structure, which could underpin more robust AE detection and CE validation.