[Paper Review] Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment

Di Jin, Zhijing Jin+2 more|arXiv (Cornell University)|Jul 27, 2019

Adversarial Robustness in Machine Learning33 references102 citations

TL;DR

TextFooler is a strong black-box adversarial attack that subtly perturbs text to flip predictions in classification and entailment tasks while preserving semantics and fluency, achieving high success with limited perturbations.

ABSTRACT

Machine learning algorithms are often vulnerable to adversarial examples that have imperceptible alterations from the original counterparts but can fool the state-of-the-art models. It is helpful to evaluate or even improve the robustness of these models by exposing the maliciously crafted adversarial examples. In this paper, we present TextFooler, a simple but strong baseline to generate natural adversarial text. By applying it to two fundamental natural language tasks, text classification and textual entailment, we successfully attacked three target models, including the powerful pre-trained BERT, and the widely used convolutional and recurrent neural networks. We demonstrate the advantages of this framework in three ways: (1) effective---it outperforms state-of-the-art attacks in terms of success rate and perturbation rate, (2) utility-preserving---it preserves semantic content and grammaticality, and remains correctly classified by humans, and (3) efficient---it generates adversarial text with computational complexity linear to the text length. *The code, pre-trained target models, and test examples are available at https://github.com/jind11/TextFooler.

Motivation & Objective

• Motivate robust evaluation of NLP models against adversarial examples.
• Propose TextFooler as a simple, strong baseline for text-based attacks in black-box settings.
• Ensure adversarial text remains semantically similar and grammatically fluent while fooling models.
• Evaluate effectiveness across diverse datasets and target architectures including BERT, CNN, and LSTM.
• Open-source the attack code and resources for benchmarking.

Proposed method

• Identify influential words in a sentence without gradients using a word-importance ranking heuristic.
• Replace high-importance words with semantically similar, grammatically correct candidates using POS-filtered synonyms and semantic similarity constraints.
• Use a sentence encoder (USE) to constrain semantic similarity and maintain meaning.
• Evaluate replacements by model confidence changes and final prediction disruption under a black-box setting.
• Conduct automatic and human evaluations across multiple NLP tasks and models to assess attack efficacy and utility preservation.
• Optionally perform adversarial training to gauge robustness improvements.

Experimental results

Research questions

• RQ1 How vulnerable are state-of-the-art NLP models (including BERT) to black-box adversarial text attacks?
• RQ2 Can an attack preserve semantic meaning and grammaticality while effectively changing predictions?
• RQ3 What are the trade-offs between perturbation rate, semantic similarity, and attack success across tasks like text classification and textual entailment?
• RQ4 Do adversarial examples transfer between different models and architectures?
• RQ5 Can adversarial training improve model robustness against such attacks?

Key findings

• TextFooler achieves high attack effectiveness with limited perturbations, frequently reducing accuracy below 15% while perturbing under 20% of words.
• The generated adversaries maintain semantic similarity and grammaticality as judged by humans and automatic measures.
• The approach is effective across multiple datasets and target models, including WordCNN, WordLSTM, and BERT, for both text classification and textual entailment.
• Word importance ranking is crucial; removing it markedly degrades attack efficacy.
• There is measurable transferability of adversarial examples between models, with higher transferability observed in entailment tasks; adversarial training can increase robustness to such attacks.