[Paper Review] Towards Crafting Text Adversarial Samples

Suranjana Samanta, Sameep Mehta|arXiv (Cornell University)|Jul 10, 2017

Adversarial Robustness in Machine LearningComputer Science9 references165 citations

TL;DR

The paper proposes a greedy, word-level method to craft semantically similar adversarial text samples by replacing, inserting, or removing words, evaluated on IMDB sentiment and Twitter gender datasets, and compares to TextFool.

ABSTRACT

Adversarial samples are strategically modified samples, which are crafted with the purpose of fooling a classifier at hand. An attacker introduces specially crafted adversarial samples to a deployed classifier, which are being mis-classified by the classifier. However, the samples are perceived to be drawn from entirely different classes and thus it becomes hard to detect the adversarial samples. Most of the prior works have been focused on synthesizing adversarial samples in the image domain. In this paper, we propose a new method of crafting adversarial text samples by modification of the original samples. Modifications of the original text samples are done by deleting or replacing the important or salient words in the text or by introducing new words in the text sample. Our algorithm works best for the datasets which have sub-categories within each of the classes of examples. While crafting adversarial samples, one of the key constraint is to generate meaningful sentences which can at pass off as legitimate from language (English) viewpoint. Experimental results on IMDB movie review dataset for sentiment analysis and Twitter dataset for gender detection show the efficiency of our proposed method.

Motivation & Objective

• Motivate the need for adversarial text samples and address the challenges of maintaining semantics and grammar.
• Propose a greedy, word-level modification method (replacement, insertion, removal) to flip classifier decisions.
• Leverage genre sub-categories and keyword sets to improve adversarial sample generation.
• Evaluate the method on IMDB sentiment and Twitter gender datasets against TextFool.
• Demonstrate robustness via retraining with adversarial samples.

Proposed method

• Compute word-wise contribution to class probability using a classifier cost gradient (FGSM-inspired).
• Rank words by contribution and iteratively modify the most impactful words.
• Build a candidate pool of replacements including synonyms, typos, and genre-specific keywords.
• Apply three modification types (removal of adverbs, insertion of adverbs before adjectives, or replacement) to create adversarial samples while preserving grammar.
• Use genre-specific keywords to improve adversarial sample quality and count.
• Retrain the classifier on adversarial samples to assess robustness.

Experimental results

Research questions

• RQ1 Can text adversarial samples be crafted with minimal, semantics-preserving edits in the text domain?
• RQ2 How effective are replacement, insertion, and removal operations at flipping sentiment or gender classification?
• RQ3 Does incorporating genre-specific keywords improve the quality and quantity of adversarial samples?
• RQ4 What is the impact of adversarial training on model robustness to text-based adversarial attacks?

Key findings

• The proposed method can synthesize semantically correct adversarial text samples for both IMDB sentiment and Twitter gender tasks.
• Including genre-specific keywords increases the number of successful adversarial samples and reduces semantic similarity slightly.
• Adversarial samples generated without genre keywords show lower perturbation counts and semantic fidelity compared to the genre-aware approach.
• Retraining CNNs with adversarial samples narrows the accuracy gap between original and adversarial test sets, indicating improved robustness.
• Compared to TextFool, the proposed method yields more adversarial samples and often higher perturbation rates, depending on configuration.
• Semantic similarity between original and adversarial texts remains high (approx. 0.92–0.99 range) in reported cases.