[Paper Review] Data Quality Matters For Adversarial Training: An Empirical Study

Chengyu Dong, Liyuan Liu+1 more|arXiv (Cornell University)|Feb 15, 2021

Adversarial Robustness in Machine Learning97 references4 citations

TL;DR

This paper demonstrates that low-quality data in training sets is a root cause of key challenges in adversarial training—robust overfitting, robustness overestimation, and the robustness-accuracy trade-off. By measuring data quality through learning stability during adversarial training, the authors show that removing low-quality examples significantly reduces these problems, leading to more reliable robustness and minimal accuracy drop.

ABSTRACT

Multiple intriguing problems are hovering in adversarial training, including robust overfitting, robustness overestimation, and robustness-accuracy trade-off. These problems pose great challenges to both reliable evaluation and practical deployment. Here, we empirically show that these problems share one common cause -- low-quality samples in the dataset. Specifically, we first propose a strategy to measure the data quality based on the learning behaviors of the data during adversarial training and find that low-quality data may not be useful and even detrimental to the adversarial robustness. We then design controlled experiments to investigate the interconnections between data quality and problems in adversarial training. We find that when low-quality data is removed, robust overfitting and robustness overestimation can be largely alleviated; and robustness-accuracy trade-off becomes less significant. These observations not only verify our intuition about data quality but may also open new opportunities to advance adversarial training.

Motivation & Objective

• To investigate the role of data quality in adversarial training, particularly its impact on robust overfitting, robustness overestimation, and the robustness-accuracy trade-off.
• To develop a data quality measurement strategy based on learning stability during adversarial training.
• To empirically demonstrate that low-quality data undermines adversarial robustness and distorts evaluation metrics.
• To show that high-quality data enables adversarial training with minimal robustness-accuracy trade-off and reduced overfitting.
• To provide a holistic, data-centric analysis of adversarial training problems, distinguishing it from prior work focused on model architecture or optimization.

Proposed method

• Propose a data quality metric based on the stability of examples during adversarial training—unstable learning indicates low-quality data.
• Partition datasets (CIFAR-10, CIFAR-100, Tiny-ImageNet) into high- and low-quality subsets using the proposed metric, ensuring class balance and equal size.
• Conduct controlled experiments comparing adversarial training on high-quality vs. low-quality subsets using PGD and TRADES with ResNet and Wide ResNet architectures.
• Use fine-tuning with targeted attacks to assess robustness loss and recognizability degradation on specific classes.
• Measure robustness overestimation via the gap between PGD-10 and Auto Attack evaluations, analyzing the role of competing classes and low-quality examples.
• Analyze the inner maximization process in adversarial attacks to explain how low-quality data reduces adversarial perturbation effectiveness.

Experimental results

Research questions

• RQ1 How does data quality, measured by learning stability, influence adversarial robustness and training dynamics?
• RQ2 To what extent does low-quality data contribute to robust overfitting in adversarial training?
• RQ3 Can low-quality data lead to robustness overestimation, and if so, under what conditions?
• RQ4 How does data quality affect the robustness-accuracy trade-off in adversarial training?
• RQ5 What mechanisms explain why low-quality data undermines adversarial robustness, particularly through feature ambiguity and competing classes?

Key findings

• Low-quality data, identified by unstable learning during adversarial training, is detrimental to adversarial robustness and can even be harmful.
• Adversarial training on low-quality data leads to significant robust overfitting, where models fail to generalize robustly across classes.
• Robustness overestimation is substantially increased when low-quality examples from two 'competing' classes (e.g., Airplane and Ship) are added to a high-quality subset, with the overestimation gap rising notably.
• When only high-quality data is used, the robustness-accuracy trade-off is minimal and only noticeable in the late training phase, unlike with low-quality data where the trade-off is pronounced.
• Fine-tuning on low-quality data causes a significant drop in robust accuracy for the targeted class, indicating loss of robust recognizability, while fine-tuning on high-quality data preserves performance.
• The mechanism of robustness overestimation is linked to ambiguous features in low-quality data, which reduce the adversary’s ability to generate effective perturbations by making features from different classes indistinguishable.