[Paper Review] Towards Evaluating the Robustness of Neural Networks Learned by Transduction

Jiefeng Chen, Xi Wu+3 more|arXiv (Cornell University)|Oct 27, 2021

Adversarial Robustness in Machine Learning35 references4 citations

TL;DR

This paper proposes Greedy Model Space Attack (GMSA), a novel adaptive attack framework for evaluating transductive-learning-based defenses in neural networks. By formulating a robust bilevel optimization objective and leveraging model space transferability, GMSA breaks defenses like DENT, RMC, and URejectron—previously resilient to AutoAttack—while demonstrating that transductive adversarial training significantly boosts robustness, suggesting a new path for defense design.

ABSTRACT

There has been emerging interest in using transductive learning for adversarial robustness (Goldwasser et al., NeurIPS 2020; Wu et al., ICML 2020; Wang et al., ArXiv 2021). Compared to traditional defenses, these defense mechanisms "dynamically learn" the model based on test-time input; and theoretically, attacking these defenses reduces to solving a bilevel optimization problem, which poses difficulty in crafting adaptive attacks. In this paper, we examine these defense mechanisms from a principled threat analysis perspective. We formulate and analyze threat models for transductive-learning based defenses, and point out important subtleties. We propose the principle of attacking model space for solving bilevel attack objectives, and present Greedy Model Space Attack (GMSA), an attack framework that can serve as a new baseline for evaluating transductive-learning based defenses. Through systematic evaluation, we show that GMSA, even with weak instantiations, can break previous transductive-learning based defenses, which were resilient to previous attacks, such as AutoAttack. On the positive side, we report a somewhat surprising empirical result of "transductive adversarial training": Adversarially retraining the model using fresh randomness at the test time gives a significant increase in robustness against attacks we consider.

Motivation & Objective

• To address the lack of principled threat models for transductive-learning-based defenses in adversarial robustness.
• To analyze the theoretical and practical challenges in crafting adaptive attacks against defenses that dynamically learn at test time.
• To propose a new attack baseline—Greedy Model Space Attack (GMSA)—that effectively undermines existing transductive defenses.
• To evaluate the robustness of transductive defenses under realistic, adaptive attack scenarios, revealing limitations in current approaches.
• To explore and validate a new defense strategy—transductive adversarial training—that shows strong empirical gains.

Proposed method

• Formulates a rigorous 1-round and multi-round threat model for transductive defenses, capturing the interaction between attacker and defender at test time.
• Proposes a robust bilevel attack objective that maximizes attack success across a neighborhood of models, relaxing the need for differentiability.
• Introduces Greedy Model Space Attack (GMSA), a greedy optimization framework that searches for adversarial examples effective across a family of models.
• Uses transferability of adversarial examples across models to approximate the bilevel attack objective, enabling practical deployment.
• Employs ensemble attacks and fresh randomness in retraining to simulate realistic test-time adaptation scenarios.
• Validates defenses using standard benchmarks (CIFAR-10, MNIST, GTSRB) and compares robust accuracy under GMSA and AutoAttack.

Experimental results

Research questions

• RQ1 How can we formally model the threat landscape for transductive-learning-based defenses in adversarial robustness?
• RQ2 What are the key theoretical and practical challenges in crafting adaptive attacks against defenses that learn at test time?
• RQ3 Can a new attack framework outperform existing baselines like AutoAttack in breaking transductive defenses?
• RQ4 Does transductive adversarial training significantly improve robustness compared to standard adversarial training?
• RQ5 What are the failure modes of existing transductive defenses like URejectron under diverse attack types?

Key findings

• GMSA successfully breaks DENT, RMC, and URejectron defenses, reducing their robust accuracy to levels comparable to standard adversarial training alone.
• On CIFAR-10, DENT's robust accuracy drops from 57.77% (under AutoAttack) to 54.12% under GMSA, indicating less than 3% improvement over adversarial training.
• Transductive adversarial training (TADV) improves robustness on MNIST from 86.61% to 94.27% and on CIFAR-10 from 45.29% to 54.12%.
• URejectron fails under CW and image corruption attacks, showing imperceptible adversarial examples can bypass it and benign perturbations can be incorrectly rejected.
• Adversarial examples generated by GMSA show poor transferability to newly retrained models under TADV, indicating the defense is robust to such attacks.
• The study reveals that transductive learning does not inherently improve robustness and that current defenses are vulnerable to adaptive, bilevel-aware attacks.