[Paper Review] Defending Pre-trained Language Models from Adversarial Word Substitutions Without Performance Sacrifice

Rongzhou Bao, Jiayi Wang+1 more|arXiv (Cornell University)|May 30, 2021

Adversarial Robustness in Machine Learning23 references4 citations

TL;DR

This paper proposes ADFAR, a compact defense framework that combines frequency-aware randomization and an auxiliary anomaly detector to defend pre-trained language models against adversarial word substitutions without sacrificing performance. By detecting adversarial inputs and applying targeted randomization only to them, ADFAR achieves superior robustness and inference speed while preserving original accuracy on non-adversarial samples.

ABSTRACT

Pre-trained contextualized language models (PrLMs) have led to strong performance gains in downstream natural language understanding tasks. However, PrLMs can still be easily fooled by adversarial word substitution, which is one of the most challenging textual adversarial attack methods. Existing defence approaches suffer from notable performance loss and complexities. Thus, this paper presents a compact and performance-preserved framework, Anomaly Detection with Frequency-Aware Randomization (ADFAR). In detail, we design an auxiliary anomaly detection classifier and adopt a multi-task learning procedure, by which PrLMs are able to distinguish adversarial input samples. Then, in order to defend adversarial word substitution, a frequency-aware randomization process is applied to those recognized adversarial input samples. Empirical results show that ADFAR significantly outperforms those newly proposed defense methods over various tasks with much higher inference speed. Remarkably, ADFAR does not impair the overall performance of PrLMs. The code is available at https://github.com/LilyNLP/ADFAR

Motivation & Objective

• To address the vulnerability of pre-trained language models (PrLMs) to adversarial word substitution attacks that maintain grammatical correctness and semantic coherence.
• To develop a defense mechanism that maintains high prediction accuracy on non-adversarial inputs, avoiding the performance degradation common in existing defense methods.
• To design an efficient, low-complexity defense framework suitable for real-world deployment with minimal computational overhead.
• To enable robustness against heuristic word substitution attacks without requiring extensive retraining or model modification.

Proposed method

• An auxiliary anomaly detection classifier is trained alongside the main PrLM via multi-task learning to identify adversarial input samples.
• Only detected adversarial inputs undergo a frequency-aware randomization process during inference, where words are replaced with more frequent synonyms to reduce adversarial impact.
• The frequency-aware strategy prioritizes replacing rare words with their more frequent synonyms, leveraging the observation that PrLMs are more robust to frequent words.
• Randomization is applied selectively—only to adversarial samples—preserving the original prediction pipeline for clean inputs to avoid performance loss.
• Synonym set construction uses a frequency threshold to filter synonyms, ensuring only semantically and syntactically plausible substitutions are considered.
• The training process incorporates frequency-aware randomization with a controlled substitution ratio to augment training data and improve robustness.

Experimental results

Research questions

• RQ1 Can a defense framework detect adversarial word substitutions without degrading performance on clean inputs?
• RQ2 How effective is frequency-aware randomization in mitigating adversarial word substitution attacks?
• RQ3 Can an auxiliary anomaly detector reliably distinguish adversarial inputs from clean ones in the context of word substitution attacks?
• RQ4 What is the optimal balance between substitution ratio and synonym set size for robustness and inference efficiency?
• RQ5 Does the combination of anomaly detection and frequency-aware randomization outperform existing defense methods in both robustness and inference speed?

Key findings

• ADFAR achieves significantly higher adversarial accuracy than prior defense methods such as those by Ye et al. (2020) and Zhou et al. (2019) across multiple NLP tasks.
• The method preserves the original prediction accuracy on non-adversarial inputs, confirming no performance sacrifice.
• The highest adversarial accuracy is achieved with a substitution ratio of 30% during inference and 25% during training, indicating optimal trade-offs between robustness and fidelity.
• Frequency-aware randomization contributes most to the defense, with performance peaks when rare words are replaced by more frequent synonyms.
• The auxiliary anomaly detector effectively identifies adversarial samples generated by current heuristic word substitution strategies, enabling targeted application of randomization.
• ADFAR demonstrates a markedly higher inference speed compared to existing defense methods, making it suitable for real-time deployment.