[Paper Review] Adversarial Robustness as a Prior for Learned Representations

Logan Engstrom, Andrew Ilyas+4 more|arXiv (Cornell University)|Jun 3, 2019

Adversarial Robustness in Machine Learning49 references76 citations

TL;DR

The paper shows that representations learned by adversarially robust networks act as a prior that yields more human-aligned, approximately invertible and easily visualizable feature embeddings, enabling semantic inversion, visualization, and manipulation.

ABSTRACT

An important goal in deep learning is to learn versatile, high-level feature representations of input data. However, standard networks' representations seem to possess shortcomings that, as we illustrate, prevent them from fully realizing this goal. In this work, we show that robust optimization can be re-cast as a tool for enforcing priors on the features learned by deep neural networks. It turns out that representations learned by robust models address the aforementioned shortcomings and make significant progress towards learning a high-level encoding of inputs. In particular, these representations are approximately invertible, while allowing for direct visualization and manipulation of salient input features. More broadly, our results indicate adversarial robustness as a promising avenue for improving learned representations. Our code and models for reproducing these results is available at https://git.io/robust-reps .

Motivation & Objective

• Demonstrate limitations of standard representations in capturing human-interpretable high-level features.
• Show that robust optimization induces priors on learned representations that align with human perception.
• Investigate whether robust representations are approximately invertible and amenable to direct visualization and manipulation.

Proposed method

• Train ResNet-50 models with standard and robust objectives on Restricted ImageNet and ImageNet.
• Use PGD-based robust optimization to minimize worst-case loss within an perturbation set.
• Compare representation properties between standard and robust models, including inversion and visualization tasks.

Experimental results

Research questions

• RQ1 Do robust representations mitigate misalignment between learned features and human-perceived features observed in standard models?
• RQ2 Are robust representations approximately invertible and amenable to direct visualization without heavy regularization?
• RQ3 Can robust representations support feature manipulation and semantically meaningful interpolation between inputs?
• RQ4 How do robust representations perform on out-of-distribution inputs with respect to inversion and visualization?

Key findings

• Robust representations are approximately invertible, yielding semantically similar inputs for nearby representations.
• Direct optimization on robust representations yields human-recognizable visualizations without regularization.
• Robust representations enable feature manipulation by adding salient features to images via first-order optimization.
• Inversion remains meaningful even for out-of-distribution inputs, suggesting features aligned with human perception.
• Interpolation in representation space between arbitrary inputs produces perceptually plausible transitions.
• Robust optimization acts as a prior that yields feature embeddings more aligned with human perception than standard models.