[Paper Review] Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks

David Stutz, Matthias Hein+1 more|arXiv (Cornell University)|Oct 14, 2019

Adversarial Robustness in Machine Learning76 references35 citations

TL;DR

CCAT biases predictions on adversarial examples toward low confidence and uses confidence-thresholding to reject them, enabling robustness to unseen attacks beyond the training threat model.

ABSTRACT

Adversarial training yields robust models against a specific threat model, e.g., $L_\infty$ adversarial examples. Typically robustness does not generalize to previously unseen threat models, e.g., other $L_p$ norms, or larger perturbations. Our confidence-calibrated adversarial training (CCAT) tackles this problem by biasing the model towards low confidence predictions on adversarial examples. By allowing to reject examples with low confidence, robustness generalizes beyond the threat model employed during training. CCAT, trained only on $L_\infty$ adversarial examples, increases robustness against larger $L_\infty$, $L_2$, $L_1$ and $L_0$ attacks, adversarial frames, distal adversarial examples and corrupted examples and yields better clean accuracy compared to adversarial training. For thorough evaluation we developed novel white- and black-box attacks directly attacking CCAT by maximizing confidence. For each threat model, we use $7$ attacks with up to $50$ restarts and $5000$ iterations and report worst-case robust test error, extended to our confidence-thresholded setting, across all attacks.

Motivation & Objective

• Motivate robust classifiers that generalize beyond a single threat model (e.g., L_infinity).
• Develop a training objective that reduces overconfidence on adversarial examples to enable extrapolation to unseen attacks.
• Enable a reject option through confidence thresholding to maintain robustness without sacrificing clean accuracy.

Proposed method

• Introduce CCAT, which trains on a convex combination of one-hot (for clean/adversarial within the seen ball) and uniform distributions (for adversarial examples) as target labels.
• Use an adaptive attack during training that maximizes confidence in any non-true class to generate challenging adversarial examples (Eq. 4).
• Define a confidence-dependent transition lambda(delta) that decays from 1 to 0 as perturbation norm grows, causing the target distribution to become more uniform with distance (Eq. 6).
• Enforce a power-transition scheme so that lambda becomes 0 for perturbations at or beyond the training epsilon, encouraging extrapolation beyond the training ball.
• Train with a 50/50 mix of clean and adversarial examples per batch, as in standard AT setups, to balance robustness and accuracy.
• Evaluate robustness using confidence-thresholded metrics and develop adaptive white/black-box attacks that explicitly maximize confidence (including within CCAT’s framework).

Experimental results

Research questions

• RQ1 Can confidence calibration via CCAT enable robustness against unseen adversarial threats not present during training?
• RQ2 Does forcing low confidence on adversarial examples within the training ball improve extrapolation to larger perturbations, other L_p norms, and corrupted inputs?
• RQ3 How does CCAT's performance compare to standard adversarial training and TRADES when a rejection option is allowed?
• RQ4 Can robustness be achieved without sacrificing clean accuracy when applying confidence-thresholding at test time?

Key findings

• CCAT improves robustness against unseen attacks (different L_p norms, larger perturbations) while preserving or improving clean accuracy compared to standard AT.
• Robustness extends to adversarial frames, distal adversarial examples, and corrupted inputs under CCAT with confidence-thresholding.
• A rejection option via confidence thresholding allows discarding low-confidence adversarial inputs, contributing to robustness beyond the training threat model.
• Adaptive, confidence-maximizing attacks are used to evaluate CCAT’s resilience; CCAT remains harder to crack under per-example worst-case evaluation.
• Experiments across MNIST, SVHN, CIFAR-10, and corrupted variants demonstrate CCAT’s advantages relative to AT, TRADES, and multi-threat-model training.