[Paper Review] Adversarial Robustness through the Lens of Causality.

Yonggang Zhang, Mingming Gong+6 more|arXiv (Cornell University)|Jun 11, 2021

Adversarial Robustness in Machine LearningComputer Science52 references20 citations

TL;DR

This paper proposes a causal framework to mitigate adversarial vulnerability in deep neural networks by modeling adversarial examples as a distribution shift driven by spurious correlations between label and style information. By aligning the natural and adversarial distributions through causal intervention, the method reduces reliance on spurious features, significantly improving robustness with state-of-the-art performance.

ABSTRACT

The adversarial vulnerability of deep neural networks has attracted significant attention in machine learning. From a causal viewpoint, adversarial attacks can be considered as a specific type of distribution change on natural data. As causal reasoning has an instinct for modeling distribution change, we propose to incorporate causality into mitigating adversarial vulnerability. However, causal formulations of the intuition of adversarial attack and the development of robust DNNs are still lacking in the literature. To bridge this gap, we construct a causal graph to model the generation process of adversarial examples and define the adversarial distribution to formalize the intuition of adversarial attacks. From a causal perspective, we find that the label is spuriously correlated with the style (content-independent) information when an instance is given. The spurious correlation implies that the adversarial distribution is constructed via making the statistical conditional association between style information and labels drastically different from that in natural distribution. Thus, DNNs that fit the spurious correlation are vulnerable to the adversarial distribution. Inspired by the observation, we propose the adversarial distribution alignment method to eliminate the difference between the natural distribution and the adversarial distribution. Extensive experiments demonstrate the efficacy of the proposed method. Our method can be seen as the first attempt to leverage causality for mitigating adversarial vulnerability.

Motivation & Objective

• To address the lack of causal understanding in adversarial robustness and explain why deep neural networks are vulnerable to adversarial examples.
• To formalize the concept of adversarial attacks as a distribution shift, particularly through spurious correlations between label and content-independent style features.
• To develop a method that mitigates adversarial vulnerability by aligning the natural and adversarial distributions using causal principles.
• To provide a principled, causality-based alternative to existing adversarial training and data augmentation techniques.

Proposed method

• Construct a causal graph to model the generation process of adversarial examples, explicitly representing the role of style (content-independent) features.
• Define the adversarial distribution as a shift in the conditional distribution of labels given style features, distinct from the natural distribution.
• Identify that models become vulnerable due to spurious correlations between labels and style information under the adversarial distribution.
• Propose an adversarial distribution alignment method that minimizes the statistical difference between the natural and adversarial conditional distributions.
• Use causal intervention to remove the spurious correlation between style and label, encouraging models to rely on content-dependent features instead.
• Train models using a regularized objective that encourages invariance to style shifts, improving robustness under adversarial perturbations.

Experimental results

Research questions

• RQ1 How can adversarial attacks be formally understood as a distribution shift in the context of causal modeling?
• RQ2 Why do deep neural networks exhibit spurious correlations between labels and style features under adversarial distributions?
• RQ3 Can causal intervention be used to eliminate spurious correlations and improve adversarial robustness?
• RQ4 How does aligning the natural and adversarial distributions improve model generalization under adversarial attacks?
• RQ5 Is there a principled, causality-driven alternative to current adversarial training methods?

Key findings

• The proposed method achieves state-of-the-art robust accuracy on standard benchmarks such as CIFAR-10 and CIFAR-100 under FGSM and PGD attacks.
• The model shows significant improvement in robustness by reducing reliance on spurious style features, as verified through ablation studies.
• The causal intervention effectively mitigates the distribution shift induced by adversarial examples, leading to more invariant representations.
• The method generalizes well across different attack types and perturbation magnitudes, demonstrating strong robustness.
• Empirical results confirm that eliminating spurious style-label correlations leads to more reliable and robust predictions under adversarial conditions.