[Paper Review] Multi-shot NAS for Discovering Adversarially Robust Convolutional Neural Architectures at Targeted Capacities.

Xuefei Ning, Junbo Zhao+4 more|arXiv (Cornell University)|Dec 22, 2020

Adversarial Robustness in Machine Learning27 references4 citations

TL;DR

This paper proposes a multi-shot NAS framework that discovers adversarially robust CNN architectures at targeted FLOPs by estimating rewards via interior extrapolation across multiple supernets, enabling capacity-aligned search. At 1560M FLOPs, MSRobNet-1560 achieves 84.8% clean accuracy and 52.9% PGD100 robust accuracy, outperforming RobNet-free under similar capacity constraints.

ABSTRACT

Convolutional neural networks (CNNs) are vulnerable to adversarial examples, and studies show that increasing the model capacity of an architecture topology (e.g., width expansion) can bring consistent robustness improvements. This reveals a clear robustness-efficiency trade-off that should be considered in architecture design. Recent studies have employed one-shot neural architecture search (NAS) to discover adversarially robust architectures. However, since the capacities of different topologies cannot be easily aligned during the search process, current one-shot NAS methods might favor topologies with larger capacity in the supernet. And the discovered topology might be sub-optimal when aligned to the targeted capacity. This paper proposes a novel multi-shot NAS method to explicitly search for adversarially robust architectures at a certain targeted capacity. Specifically, we estimate the reward at the targeted capacity using interior extra-polation of the rewards from multiple supernets. Experimental results demonstrate the effectiveness of the proposed method. For instance, at the targeted FLOPs of 1560M, the discovered MSRobNet-1560 (clean 84.8%, PGD100 52.9%) outperforms the recent NAS-discovered architecture RobNet-free (clean 82.8%, PGD100 52.6%) with similar FLOPs. Codes are available at this https URL.

Motivation & Objective

• To address the robustness-efficiency trade-off in CNN architecture design by enabling targeted capacity search.
• To overcome the limitation of one-shot NAS in favoring high-capacity topologies due to supernet capacity misalignment.
• To develop a method that estimates robustness rewards at a specific target capacity using multiple supernets.
• To discover architectures that are both robust and efficient at predefined computational budgets.
• To improve upon existing NAS-discovered models like RobNet-free by achieving better robustness and accuracy at similar FLOPs.

Proposed method

• The method employs multiple supernets, each with different capacity levels, to enable capacity-aware architecture search.
• It uses interior extrapolation to estimate the reward (e.g., robust accuracy) at a target capacity not directly present in any single supernet.
• Reward estimation is based on the performance trends observed across multiple supernets with varying widths or depths.
• The search process selects architecture paths that maximize the estimated reward at the target FLOPs.
• The approach enables fair comparison and optimization across architectures with aligned capacities.
• The final architecture is trained and evaluated at the target FLOPs to validate robustness and accuracy.

Experimental results

Research questions

• RQ1 Can multi-shot NAS improve the discovery of adversarially robust CNNs at specific, targeted FLOPs?
• RQ2 Does interior extrapolation of rewards across multiple supernets lead to better capacity-aligned architecture search than one-shot NAS?
• RQ3 Can the proposed method outperform existing NAS-discovered models like RobNet-free in both clean and robust accuracy at similar FLOPs?
• RQ4 How does the performance of the discovered architecture vary when aligned to a precise computational budget?
• RQ5 To what extent does capacity misalignment in one-shot NAS bias the search toward higher-capacity, sub-optimal topologies?

Key findings

• At 1560M FLOPs, MSRobNet-1560 achieves 84.8% clean accuracy and 52.9% PGD100 robust accuracy, outperforming RobNet-free.
• RobNet-free achieves 82.8% clean accuracy and 52.6% PGD100 robust accuracy at similar FLOPs, confirming the performance gain of the proposed method.
• The multi-shot NAS method successfully estimates rewards at target capacities via interior extrapolation, enabling fair and accurate search.
• The approach mitigates the bias toward high-capacity topologies observed in one-shot NAS by aligning capacity during search.
• The discovered architecture demonstrates consistent robustness improvements across different FLOP levels, validating the method's scalability.
• The method enables the discovery of robust architectures that are both efficient and effective at predefined computational budgets.