[Paper Review] Fairwashing Explanations with Off-Manifold Detergent

Christopher J. Anders, Plamen Plasiliev+3 more|arXiv (Cornell University)|Jul 20, 2020

Explainable Artificial Intelligence (XAI)35 citations

TL;DR

The paper proves that explanation maps for classifiers can be arbitrarily manipulated without changing outputs, and proposes tangent-space projections to make explanations more robust against such manipulations. It provides theoretical proofs using differential geometry and validates them experimentally across tasks and architectures.

ABSTRACT

Explanation methods promise to make black-box classifiers more transparent. As a result, it is hoped that they can act as proof for a sensible, fair and trustworthy decision-making process of the algorithm and thereby increase its acceptance by the end-users. In this paper, we show both theoretically and experimentally that these hopes are presently unfounded. Specifically, we show that, for any classifier $g$, one can always construct another classifier $ ilde{g}$ which has the same behavior on the data (same train, validation, and test error) but has arbitrarily manipulated explanation maps. We derive this statement theoretically using differential geometry and demonstrate it experimentally for various explanation methods, architectures, and datasets. Motivated by our theoretical insights, we then propose a modification of existing explanation methods which makes them significantly more robust.

Motivation & Objective

• Motivate the risk that standard explanations may not faithfully reflect model decisions.
• Theoretically prove manipulability of explanation maps under manifold assumptions.
• Demonstrate experimentally that manipulated explanations can diverge from originals while preserving performance.
• Propose and validate a robust explanation method based on projecting explanations onto the data manifold's tangent space.

Proposed method

• Use differential geometry to show that explanations can be extended off the data manifold to match arbitrary targets while keeping model behavior on the data unchanged.
• Prove that for a d-dimensional data manifold S embedded in R^D, explanations orthogonal to S are underdetermined and can be controlled to reproduce any target explanation within an epsilon = d/D bound.
• Define tangent-space-projected (tsp) explanations by projecting gradients onto the data manifold’s tangent space.
• Provide practical methods to estimate the tangent space (Hyperplane method and Autoencoder-based method) for tsp explanations in high dimensions.
• Experimentally manipulate explanations for logistic regression and deep networks on MNIST, FashionMNIST, CIFAR-10, showing robust manipulation and evaluation via SSIM, PCC, MSE, and KL divergence.

Experimental results

Research questions

• RQ1 Can explanations of a classifier be manipulated without changing its predictions on the data manifold?
• RQ2 How does the dimensionality of the data manifold relative to the embedding space affect explanation manipulability?
• RQ3 Do tangent-space-projected explanations offer increased robustness against manipulation?
• RQ4 How can tangent-space projections be estimated in practice for complex datasets and models?
• RQ5 Do tsp explanations generalize across explanation types (gradient, x ∘ Grad, Integrated Gradients, LRP) and tasks?

Key findings

• Explanations can be manipulated to closely match a target explanation with high similarity (SSIM around 0.8) while the model’s outputs remain nearly identical (output MSE around 1e-3).
• The manipulation affects directions orthogonal to the data manifold; gradients tangential to the manifold remain consistent between original and manipulated models.
• Tangent-space-projected explanations are significantly more robust to manipulation, showing high similarity between original and manipulated explanations when measured on the data manifold.
• For flat submanifolds with logistic regression, modifying the decision boundary in normal directions leaves tsp explanations identical (Pw-projected gradient).
• Two practical methods to estimate tangent spaces are proposed: a hyperplane method using k-nearest neighbors and an autoencoder-based method using SVD of the Jacobian, enabling tsp explanations on various datasets.