[Paper Review] Robustness Verification for Transformers

Zhouxing Shi, Huan Zhang+3 more|arXiv (Cornell University)|Feb 16, 2020

Adversarial Robustness in Machine LearningComputer Science50 references35 citations

TL;DR

This paper introduces the first robustness verification method for Transformer models with self-attention, deriving tight linear bounds and combining forward and backward bound propagation to certify predictions under input perturbations.

ABSTRACT

Robustness verification that aims to formally certify the prediction behavior of neural networks has become an important tool for understanding model behavior and obtaining safety guarantees. However, previous methods can usually only handle neural networks with relatively simple architectures. In this paper, we consider the robustness verification problem for Transformers. Transformers have complex self-attention layers that pose many challenges for verification, including cross-nonlinearity and cross-position dependency, which have not been discussed in previous works. We resolve these challenges and develop the first robustness verification algorithm for Transformers. The certified robustness bounds computed by our method are significantly tighter than those by naive Interval Bound Propagation. These bounds also shed light on interpreting Transformers as they consistently reflect the importance of different words in sentiment analysis.

Motivation & Objective

• Motivate robustness verification as a tool for safety and interpretability in neural networks, especially for complex architectures.
• Develop a verification algorithm capable of handling Transformer's self-attention to certify predictions under perturbations.
• Produce bounds that are tighter than naive IBP and offer interpretable insights into word importance in NLP tasks.
• Demonstrate applicability on sentiment analysis datasets and analyze efficiency and tightness of the proposed method.

Proposed method

• Adopt a linear-relaxation framework to propagate lower and upper bounds of neuron outputs under input perturbations.
• Handle self-attention by deriving closed-form linear bounds for dot products, softmax, and weighted sums to manage cross-nonlinearity.
• Introduce a forward bound propagation process for self-attention to reduce complexity, and a backward process for tighter bounds.
• Combine forward and backward bound propagation to achieve O(m^2 n) bound matrices, significantly reducing complexity from O(m^2 n^2).
• Backpropagate bounds through linear transformations and unary nonlinearities, extending CROWN-like bounds to Transformers.

Experimental results

Research questions

• RQ1 Can robustness of Transformer-based models be certified under small perturbations to input sequences?
• RQ2 How can linear bounds be efficiently propagated through self-attention, which involves cross-position dependencies and cross-nonlinearities?
• RQ3 Do the resulting certified bounds meaningfully reflect the importance of input words in tasks like sentiment analysis?

Key findings

• Certified robustness bounds for Transformer models are significantly tighter than those from Interval Bound Propagation (IBP).
• The forward-bound and backward-bound combination yields competitive tightness with reduced computation time compared to fully backward propagation.
• The method scales to small Transformer variants and yields bounds that correlate with word importance in sentiment analysis.
• Experimental results on Yelp and SST datasets demonstrate meaningful, interpretable robustness certificates without relying on large pre-trained models.
• Bounds show reasonable gap relative to upper bounds, indicating practical utility similar to verification for simpler networks.