[Paper Review] Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemble

Yi Zhou, Xiaoqing Zheng+3 more|arXiv (Cornell University)|Jun 20, 2020

Adversarial Robustness in Machine Learning46 references31 citations

TL;DR

Proposes Dirichlet Neighborhood Ensemble (DNE), a randomized smoothing defense for NLP that trains on virtual sentences within word synonym convex hulls and ensembles predictions to defend substitution-based adversaries.

ABSTRACT

Despite neural networks have achieved prominent performance on many natural language processing (NLP) tasks, they are vulnerable to adversarial examples. In this paper, we propose Dirichlet Neighborhood Ensemble (DNE), a randomized smoothing method for training a robust model to defense substitution-based attacks. During training, DNE forms virtual sentences by sampling embedding vectors for each word in an input sentence from a convex hull spanned by the word and its synonyms, and it augments them with the training data. In such a way, the model is robust to adversarial attacks while maintaining the performance on the original clean data. DNE is agnostic to the network architectures and scales to large models for NLP applications. We demonstrate through extensive experimentation that our method consistently outperforms recently proposed defense methods by a significant margin across different network architectures and multiple data sets.

Motivation & Objective

• Address vulnerability of NLP models to adversarial word substitutions.
• Develop a defense that is model-agnostic and scalable to large NLP architectures.
• Introduce a convex-hull based randomization (Dirichlet sampling) to create robust virtual inputs for training and inference.
• Combine Dirichlet sampling with adversarial training to better explore perturbation regions within synonym neighborhoods.

Proposed method

• Form virtual sentences by sampling embeddings from the convex hull of a word and its synonyms.
• Use Dirichlet-distributed weights to sample points in the convex hull, controlling contribution of 1-hop and expanded 2-hop neighbors.
• Expand the hull to include 1-hop and 2-hop neighbors to cover synonym neighborhoods during training.
• Train the base classifier with virtual examples via maximum likelihood objectives (negative cross-entropy) with virtual data augmentation.
• Apply adversarial training to search for worst-case perturbations inside the convex hull using gradient-guided updates.
• At inference, perform Monte Carlo sampling of virtual inputs and ensemble predictions with CBW-D weighting.

Experimental results

Research questions

• RQ1 How can we guarantee robustness of NLP models to word-substitution adversaries within synonym neighborhoods?
• RQ2 Does training with virtual sentences drawn from synonym convex hulls improve robustness without sacrificing clean accuracy?
• RQ3 Can ensemble predictions over Dirichlet-sampled variants enhance detection of adversarial inputs?
• RQ4 Is the method scalable to large architectures like BERT and applicable across tasks like text classification and natural language inference?
• RQ5 What impact do Dirichlet parameters and hull expansions have on robustness and clean performance?

Key findings

• DNE consistently outperforms adversarial training and interval bound propagation baselines on IMDB, AGNEWS, and SNLI across architectures.
• DNE achieves higher robust accuracy with limited clean-data loss compared to competing defenses in most settings.
• Expanding to 1-hop and 2-hop neighbors and using coordinated embedding updates improves robustness versus vertex-wise updates.
• Dirichlet sampling and ensemble inference enable more reliable handling of word-substitution perturbations and scale to large models like BERT.
• Ablation studies show the importance order of expansion, adversarial training, coordinated updates, and ensemble.