[Paper Review] Towards Robust Explanations for Deep Neural Networks

Ann-Kathrin Dombrowski, Christopher J. Anders+2 more|arXiv (Cornell University)|Dec 18, 2020

Explainable Artificial Intelligence (XAI)59 references4 citations

TL;DR

This paper proposes a unified theoretical framework to enhance the robustness of deep neural network explanations against input-level manipulations. By integrating weight decay, smoothed ReLU-like activations (Softplus), and Hessian norm minimization during training, the authors significantly reduce explanation sensitivity to small input perturbations, achieving up to 30% higher Pearson correlation between original and perturbed saliency maps on CIFAR-10.

ABSTRACT

Explanation methods shed light on the decision process of black-box classifiers such as deep neural networks. But their usefulness can be compromised because they are susceptible to manipulations. With this work, we aim to enhance the resilience of explanations. We develop a unified theoretical framework for deriving bounds on the maximal manipulability of a model. Based on these theoretical insights, we present three different techniques to boost robustness against manipulation: training with weight decay, smoothing activation functions, and minimizing the Hessian of the network. Our experimental results confirm the effectiveness of these approaches.

Motivation & Objective

• Address the vulnerability of deep learning explanations to small, imperceptible input perturbations that drastically alter explanation maps.
• Identify that high curvature in the model's decision boundary increases susceptibility to explanation manipulation.
• Develop a theoretical framework to bound the maximal change in explanation maps under input perturbations.
• Propose and validate three practical training techniques to enhance explanation robustness without altering inference.
• Demonstrate that robust explanations are essential for trustworthy AI in high-stakes domains like healthcare and credit assessment.

Proposed method

• Derive a unified theoretical framework linking explanation manipulability to the Hessian of the network with respect to input, enabling bounds on maximal explanation change.
• Apply weight decay during training to reduce model curvature and stabilize gradients, thereby minimizing explanation shifts under input noise.
• Replace ReLU activations with Softplus functions to smooth non-differentiable kinks, reducing sensitivity to input perturbations.
• Minimize the Hessian norm of the network at training data points via a differentiable regularization term, directly flattening the loss landscape locally.
• Use Monte Carlo sampling to efficiently approximate the Hessian norm during training, enabling scalable optimization.
• Combine all three techniques in a single training pipeline to maximize robustness across diverse explanation methods and noise types.

Experimental results

Research questions

• RQ1 To what extent can the maximal change in explanation maps be theoretically bounded under small input perturbations?
• RQ2 How do weight decay, activation smoothing, and Hessian minimization individually affect the robustness of saliency maps?
• RQ3 What is the combined effect of applying multiple robustness techniques on explanation stability across different noise distributions?
• RQ4 How does explanation robustness correlate with Hessian norm and weight norm reduction in trained models?
• RQ5 Can the proposed methods maintain high classification accuracy while significantly improving explanation consistency under adversarial input perturbations?

Key findings

• The proposed theoretical framework successfully bounds the maximal change in explanation maps due to input perturbations, providing a principled basis for robustness.
• Training with weight decay reduces both weight norms and Hessian norms, leading to a 25% increase in explanation similarity (measured by PCC) under Gaussian noise.
• Softplus activation smoothing with β=10 improves robustness by 20% compared to ReLU, especially under Laplacian and Gaussian noise.
• Hessian norm minimization achieves the strongest robustness gains, increasing PCC by up to 30% on CIFAR-10 when combined with other methods.
• The combination of all three techniques yields the most stable explanations, with PCC values consistently above 0.95 across all noise levels and types.
• Salt-pepper noise, though more disruptive, still shows significant robustness improvement when using the proposed training techniques, particularly with weight decay and Hessian minimization.