[Paper Review] Attribute-Guided Adversarial Training for Robustness to Natural Perturbations

Tejas Gokhale, Rushil Anirudh+4 more|arXiv (Cornell University)|Dec 3, 2020

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper proposes Attribute-Guided Adversarial Training (AGAT), a novel robust deep learning method that enhances model generalization by generating adversarial perturbations in semantic attribute space—such as object position, shape, or geometric transformations—rather than pixel-level norm-bounded perturbations. AGAT achieves state-of-the-art robustness on MNIST-RTS, CIFAR10-C, and a new CLEVR-Singles benchmark, outperforming prior methods in handling natural, semantics-preserving perturbations without requiring test-domain data.

ABSTRACT

While existing work in robust deep learning has focused on small pixel-level norm-based perturbations, this may not account for perturbations encountered in several real-world settings. In many such cases although test data might not be available, broad specifications about the types of perturbations (such as an unknown degree of rotation) may be known. We consider a setup where robustness is expected over an unseen test domain that is not i.i.d. but deviates from the training domain. While this deviation may not be exactly known, its broad characterization is specified a priori, in terms of attributes. We propose an adversarial training approach which learns to generate new samples so as to maximize exposure of the classifier to the attributes-space, without having access to the data from the test domain. Our adversarial training solves a min-max optimization problem, with the inner maximization generating adversarial perturbations, and the outer minimization finding model parameters by optimizing the loss on adversarial perturbations generated from the inner maximization. We demonstrate the applicability of our approach on three types of naturally occurring perturbations -- object-related shifts, geometric transformations, and common image corruptions. Our approach enables deep neural networks to be robust against a wide range of naturally occurring perturbations. We demonstrate the usefulness of the proposed approach by showing the robustness gains of deep neural networks trained using our adversarial training on MNIST, CIFAR-10, and a new variant of the CLEVR dataset.

Motivation & Objective

• To address the limitation of existing robustness methods that focus only on small, pixel-level perturbations, which do not reflect real-world distribution shifts.
• To develop a training framework that improves model robustness under unseen, natural perturbations that are not captured by standard ℓp-norm bounds.
• To enable robustness against semantics-preserving attribute shifts—such as object position, shape, or geometric transformations—without requiring access to test-domain data.
• To create a new benchmark, CLEVR-Singles, for evaluating robustness to semantic shifts in visual reasoning tasks.
• To demonstrate the generalizability of the approach across diverse perturbation types: object-level shifts, geometric transformations, and image corruptions.

Proposed method

• AGAT formulates a min-max optimization problem: the inner maximization generates adversarial attribute perturbations, while the outer minimization optimizes model parameters to minimize the loss on these perturbed samples.
• The method uses surrogate attribute functions to define perturbations in semantic space—e.g., spatial transformer networks for geometric transformations or conditional GANs for object-level shifts.
• Perturbations are generated by optimizing over attribute parameters (e.g., rotation angle, object size, blur intensity) rather than pixel values, enabling control over natural, real-world variations.
• The approach is flexible and can be applied with different surrogate functions, including those for synthetic image corruptions (e.g., blur, noise, weather effects).
• It trains models using adversarial examples synthesized from attribute space, improving robustness without requiring any test-domain data.
• The method integrates with standard classification heads and loss functions (e.g., categorical cross-entropy), making it compatible with existing deep learning pipelines.

Experimental results

Research questions

• RQ1 Can adversarial training guided by semantic attributes improve robustness to natural, real-world perturbations beyond ℓp-norm bounded noise?
• RQ2 How effective is attribute-guided adversarial training in defending against geometric transformations such as rotation and translation?
• RQ3 Can AGAT generalize to image corruptions like blur, noise, and weather effects without relying on pixel-space perturbations?
• RQ4 Does training with only specific attribute surrogates (e.g., blur-only or noise-only) yield performance gains comparable to full-attribute methods?
• RQ5 To what extent can AGAT mitigate distribution shift in visual reasoning tasks involving semantic object variations?

Key findings

• On the CIFAR10-C benchmark, AGAT achieved a mean accuracy of 72.3%, outperforming all prior baselines, with significant gains on weather (65.8%), blur (74.1%), and digital (71.6%) corruption categories.
• The AGAT model trained with only noise as a surrogate achieved 62.4% accuracy on the noise category, surpassing all previous pixel-level methods, including ALP.
• On the MNIST-RTS benchmark, AGAT achieved 77.8% accuracy, demonstrating strong robustness to geometric transformations such as rotation and translation.
• On the CLEVR-Singles benchmark, AGAT achieved 89.3% accuracy, showing state-of-the-art performance in handling semantic shifts like object position and shape variation.
• The ablation study showed that models trained with only blur or only noise as surrogates still outperformed existing pixel-level methods, proving the effectiveness of attribute-guided perturbations.
• The method generalizes across diverse perturbation types and maintains high performance even when only a subset of attributes is used during training.