[Paper Review] Are Vision Transformers Robust to Patch Perturbations?

Jindong Gu, Volker Tresp+1 more|arXiv (Cornell University)|Nov 20, 2021

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper investigates the robustness of Vision Transformers (ViT) to patch-wise perturbations, revealing that ViTs are more robust to natural corruptions but significantly more vulnerable to adversarial patches than ResNets. The authors attribute this to the self-attention mechanism, which can ignore natural corruptions but is easily misled by adversarial ones, and propose Smoothed Attention via temperature scaling to improve robustness by discouraging attention concentration on single patches.

ABSTRACT

Recent advances in Vision Transformer (ViT) have demonstrated its impressive performance in image classification, which makes it a promising alternative to Convolutional Neural Network (CNN). Unlike CNNs, ViT represents an input image as a sequence of image patches. The patch-based input image representation makes the following question interesting: How does ViT perform when individual input image patches are perturbed with natural corruptions or adversarial perturbations, compared to CNNs? In this work, we study the robustness of ViT to patch-wise perturbations. Surprisingly, we find that ViTs are more robust to naturally corrupted patches than CNNs, whereas they are more vulnerable to adversarial patches. Furthermore, we discover that the attention mechanism greatly affects the robustness of vision transformers. Specifically, the attention module can help improve the robustness of ViT by effectively ignoring natural corrupted patches. However, when ViTs are attacked by an adversary, the attention mechanism can be easily fooled to focus more on the adversarially perturbed patches and cause a mistake. Based on our analysis, we propose a simple temperature-scaling based method to improve the robustness of ViT against adversarial patches. Extensive qualitative and quantitative experiments are performed to support our findings, understanding, and improvement of ViT robustness to patch-wise perturbations across a set of transformer-based architectures.

Motivation & Objective

• To investigate how Vision Transformers (ViT) perform under patch-wise perturbations compared to Convolutional Neural Networks (CNNs).
• To understand the role of the self-attention mechanism in ViT's robustness to natural and adversarial patch corruptions.
• To develop a method that improves ViT's robustness against adversarial patch attacks based on insights from attention behavior.
• To compare the robustness of various ViT variants, CNNs, and hybrid architectures under different patch attack settings.

Proposed method

• Conduct a fair comparison of ViT and ResNet under patch-wise natural corruptions and adversarial perturbations using standard benchmarks.
• Visualize attention maps using rollout attention to analyze how ViT focuses on patches under different perturbation types.
• Propose Smoothed Attention, a temperature-scaling technique applied to the softmax in the self-attention mechanism to distribute attention weights more evenly.
• Apply temperature scaling to discourage any single patch from dominating the attention computation in subsequent layers.
• Evaluate robustness across multiple architectures including DeiT, Swin Transformers, LeViT, and MLP-Mixer under various attack settings.
• Test the method under imperceptible, targeted, and ViT-agnostic patch attacks to validate generalization and robustness improvements.

Experimental results

Research questions

• RQ1 How does ViT's robustness to patch-wise natural corruptions compare to that of ResNet?
• RQ2 Why is ViT more vulnerable to adversarial patch attacks despite its robustness to natural corruptions?
• RQ3 To what extent does the self-attention mechanism contribute to ViT's robustness or vulnerability under patch perturbations?
• RQ4 Can temperature scaling in the attention mechanism effectively improve ViT's robustness against adversarial patch attacks?
• RQ5 How does the architectural design of different ViT variants (e.g., Swin, MLP-Mixer) affect their robustness to patch attacks?

Key findings

• ViT is more robust than ResNet to naturally corrupted patches, with attention mechanisms effectively ignoring corrupted regions.
• ViT is significantly more vulnerable than ResNet to adversarial patch attacks, achieving a fooling rate of 61.5% on DeiT-small under standard patch attacks.
• The self-attention mechanism in ViT can be easily manipulated to focus on adversarial patches, leading to misclassification, as shown in attention map visualizations.
• Smoothed Attention via temperature scaling improves ViT's robustness against adversarial patches, with notable gains on DeiT and LeViT, though minimal improvement on Swin Transformers due to their hierarchical design.
• Under imperceptible patch attacks (bounded by 8/225), DeiT-tiny still shows higher vulnerability (11.2% fooling rate) than ResNet18 (2.9%).
• The minimum number of iterations required to fool DeiT-tiny (65) is significantly lower than for ResNet18 (342), confirming faster attack success on ViT.