[Paper Review] On the Effect of Low-Rank Weights on Adversarial Robustness of Neural Networks

Peter Langenberg, Emilio Rafael Balda+2 more|arXiv (Cornell University)|Jan 29, 2019

Adversarial Robustness in Machine Learning24 references6 citations

TL;DR

This paper investigates the role of low-rank and sparse weight matrices in improving adversarial robustness of deep neural networks. By applying nuclear norm and L1 regularization to promote low-rank and sparse structures, the authors demonstrate significant improvements in robustness against FGSM and PGD attacks, particularly when both properties are jointly optimized, though performance still falls short of adversarial training.

ABSTRACT

Recently, there has been an abundance of works on designing Deep Neural Networks (DNNs) that are robust to adversarial examples. In particular, a central question is which features of DNNs influence adversarial robustness and, therefore, can be to used to design robust DNNs. In this work, this problem is studied through the lens of compression which is captured by the low-rank structure of weight matrices. It is first shown that adversarial training tends to promote simultaneously low-rank and sparse structure in the weight matrices of neural networks. This is measured through the notions of effective rank and effective sparsity. In the reverse direction, when the low rank structure is promoted by nuclear norm regularization and combined with sparsity inducing regularizations, neural networks show significantly improved adversarial robustness. The effect of nuclear norm regularization on adversarial robustness is paramount when it is applied to convolutional neural networks. Although still not competing with adversarial training, this result contributes to understanding the key properties of robust classifiers.

Motivation & Objective

• To investigate whether low-rank and sparse weight structures contribute to adversarial robustness in deep neural networks.
• To determine if promoting these structural properties through regularization can improve robustness without full adversarial training.
• To analyze the interplay between effective rank and effective sparsity in weight matrices during adversarial training.
• To evaluate whether joint regularization on rank and sparsity yields better robustness than individual regularization.

Proposed method

• The authors use effective rank and effective sparsity as metrics to quantify the low-rank and sparse structure of weight matrices in trained networks.
• Nuclear norm regularization is applied to promote low-rank structure in convolutional and fully connected layers.
• L1 regularization is used to induce sparsity in the weight matrices.
• A joint regularization strategy combines nuclear norm and L1 penalties on selected layers to simultaneously promote low-rank and sparse structures.
• Experiments are conducted on MNIST and Fashion-MNIST using FGSM and PGD attacks with ε = 0.05 to evaluate fooling ratios.
• Weight matrices are visualized post-training to qualitatively confirm the emergence of low-rank and sparse patterns.

Experimental results

Research questions

• RQ1 Does adversarial training naturally lead to low-rank and sparse weight matrices in deep neural networks?
• RQ2 Can explicit regularization to promote low-rank and sparse weight structures improve adversarial robustness?
• RQ3 Is the combination of low-rank and sparse regularization more effective than applying either regularization alone?
• RQ4 How do effective rank and effective sparsity correlate with robustness under FGSM and PGD attacks?
• RQ5 To what extent can low-rank and sparse weight structures replicate the robustness achieved by adversarial training?

Key findings

• Adversarial training naturally induces both low-rank and sparse weight structures, as evidenced by reductions in effective rank and effective sparsity.
• Applying nuclear norm regularization to convolutional layers significantly improves robustness against PGD and FGSM attacks.
• Joint regularization with both nuclear norm and L1 penalties leads to better robustness than individual regularization, especially on convolutional layers.
• Even L1 regularization alone reduces effective rank, indicating a hidden interplay between sparsity and low-rank structure.
• Visualizations confirm that both low-rank and sparse patterns emerge in weight matrices after regularization, particularly in the first and second convolutional filters.
• Despite improvements, the robustness of regularized models remains below that of models trained with full adversarial training, indicating additional factors are at play.