[Paper Review] T3: Tree-Autoencoder Constrained Adversarial Text Generation for Targeted Attack

Boxin Wang, Hengzhi Pei+4 more|arXiv (Cornell University)|Dec 22, 2019

Topic Modeling41 references4 citations

TL;DR

T3 proposes a tree-autoencoder-based framework for targeted adversarial text generation that embeds text into a continuous space to enable gradient-based optimization, then decodes using tree-grammar rules to maintain syntactic and semantic coherence. It achieves high attack success rates across NLP tasks like sentiment analysis and QA while preserving human readability and enabling black-box transfer attacks.

ABSTRACT

Adversarial attacks against natural language processing systems, which perform seemingly innocuous modifications to inputs, can induce arbitrary mistakes to the target models. Though raised great concerns, such adversarial attacks can be leveraged to estimate the robustness of NLP models. Compared with the adversarial example generation in continuous data domain (e.g., image), generating adversarial text that preserves the original meaning is challenging since the text space is discrete and non-differentiable. To handle these challenges, we propose a target-controllable adversarial attack framework T3, which is applicable to a range of NLP tasks. In particular, we propose a tree-based autoencoder to embed the discrete text data into a continuous representation space, upon which we optimize the adversarial perturbation. A novel tree-based decoder is then applied to regularize the syntactic correctness of the generated text and manipulate it on either sentence (T3(Sent)) or word (T3(Word)) level. We consider two most representative NLP tasks: sentiment analysis and question answering (QA). Extensive experimental results and human studies show that T3 generated adversarial texts can successfully manipulate the NLP models to output the targeted incorrect answer without misleading the human. Moreover, we show that the generated adversarial texts have high transferability which enables the black-box attacks in practice. Our work sheds light on an effective and general way to examine the robustness of NLP models. Our code is publicly available at https://github.com/AI-secure/T3/.

Motivation & Objective

• To address the challenge of generating targeted adversarial text that maintains semantic meaning and syntactic correctness while evading human detection.
• To overcome the limitations of discrete, non-differentiable text space in adversarial attack generation, especially compared to continuous domains like images.
• To develop a general-purpose, target-controllable attack framework applicable across diverse NLP tasks such as sentiment analysis and question answering.
• To evaluate the robustness of state-of-the-art NLP models, particularly BERT, under targeted adversarial settings.
• To investigate the transferability of adversarial examples for practical black-box attack scenarios in NLP.

Proposed method

• A tree-based autoencoder is pretrained on a large corpus to map discrete text into a continuous semantic embedding space, enabling gradient-based optimization.
• Adversarial perturbations are applied in the continuous embedding space at either word level (T3(Word)) or sentence level (T3(Sent)), leveraging the hierarchical structure of dependency trees.
• A tree-based decoder reconstructs the adversarial text from the perturbed embedding using grammar rules, preserving syntactic structure and semantic content.
• An iterative optimization process adjusts the perturbation to maximize attack success rate while minimizing detectable changes.
• The framework supports both in-place and insertional attacks, with the latter adding adversarial sentences to original inputs.
• Transferability is leveraged for black-box attacks by generating adversarial examples on one model and transferring them to others with similar architectures.

Experimental results

Research questions

• RQ1 Which level of tree hierarchy—word or sentence—yields more effective and more natural adversarial attacks?
• RQ2 Can targeted attacks be successfully applied to general NLP tasks such as sentiment classification and question answering with high success rates?
• RQ3 To what extent do adversarial examples transfer across models, especially in black-box settings?
• RQ4 How does model architecture and attention mechanism (e.g., BERT’s long-range attention) affect the effectiveness of adversarial attacks?
• RQ5 Do adversarial examples significantly degrade human performance, or remain undetectable to human readers?

Key findings

• T3(Word) achieves the highest attack success rate across multiple tasks, though T3(Sent) produces more grammatically correct and human-undetectable adversarial sentences.
• Adversarial examples generated by T3 successfully manipulate NLP models to output targeted incorrect answers in both sentiment classification and QA tasks without misleading human annotators.
• Transferability of adversarial examples is strong, especially when transferring from stronger models (e.g., BERT-QA) to weaker ones (e.g., BiDAF), with higher success rates than reverse transfer.
• Adversarial examples show higher transferability among models with similar architectures than across different architectures.
• BERT models exhibit significant performance drops under adversarial attacks, indicating they are not robust despite their state-of-the-art accuracy.
• Attacking BERT by appending adversarial sentences at the beginning or end of a paragraph is more effective than inserting them in the middle, due to BERT’s attention bias toward sentence boundaries.