[Paper Review] From Hero to Zéroe: A Benchmark of Low-Level Adversarial Attacks

Steffen Eger, Yannik Benz|arXiv (Cornell University)|Oct 12, 2020

Adversarial Robustness in Machine Learning29 references4 citations

TL;DR

This paper introduces Zéroe, the first large-scale benchmark of low-level adversarial attacks on NLP models, focusing on character-level perturbations such as typos, visual substitutions, and phonetic variations that mimic real-world user behavior. It demonstrates that RoBERTa, despite its robustness, fails significantly under these attacks, highlighting a critical vulnerability in state-of-the-art models to human-plausible, low-level perturbations.

ABSTRACT

Adversarial attacks are label-preserving modifications to inputs of machine learning classifiers designed to fool machines but not humans. Natural Language Processing (NLP) has mostly focused on high-level attack scenarios such as paraphrasing input texts. We argue that these are less realistic in typical application scenarios such as in social media, and instead focus on low-level attacks on the character-level. Guided by human cognitive abilities and human robustness, we propose the first large-scale catalogue and benchmark of low-level adversarial attacks, which we dub Zéroe, encompassing nine different attack modes including visual and phonetic adversaries. We show that RoBERTa, NLP's current workhorse, fails on our attacks. Our dataset provides a benchmark for testing robustness of future more human-like NLP models.

Motivation & Objective

• To address the lack of realistic, low-level adversarial attack benchmarks in NLP that reflect real-world user behavior on platforms like social media.
• To identify and catalog nine distinct character-level attack modes—such as typos, visual substitutions, and phonetic shifts—based on human cognitive constraints and perceptual robustness.
• To evaluate the robustness of RoBERTa, a leading NLP model, against these attacks and demonstrate its failure under plausible adversarial conditions.
• To provide a standardized benchmark dataset and code to enable future research into more human-like, robust NLP models.

Proposed method

• The authors define and implement nine low-level attack modes: inner-shuffle, full-shuffle, intrude, disemvowel, truncate, segment, typo, natural noise, phonetic, and visual, each modifying input text at the character level.
• Each attack type is applied systematically to a diverse set of input sentences, preserving semantic meaning while altering orthographic form to mimic human errors or obfuscation.
• The benchmark is constructed using real linguistic and perceptual constraints, ensuring that perturbations remain readable and plausible to humans.
• RoBERTa is evaluated on its zero-shot classification performance across all attack types, with results compared against random baseline and clean inputs.
• Adversarial training is applied using the Zéroe dataset to assess whether model robustness can be improved, using standard training protocols.
• The benchmark is released publicly with code and data at https://github.com/yannikbenz/zeroe to support reproducibility and future research.

Experimental results

Research questions

• RQ1 How do low-level, character-level adversarial attacks—such as typos, visual substitutions, and phonetic shifts—affect the performance of state-of-the-art NLP models like RoBERTa?
• RQ2 To what extent are RoBERTa and similar models vulnerable to attacks that mimic realistic human behaviors in social media or spam contexts?
• RQ3 Can standard adversarial training techniques effectively restore model robustness when exposed to a diverse, human-plausible set of low-level perturbations?
• RQ4 Are there specific attack types that cause disproportionately large performance drops in RoBERTa, indicating structural weaknesses in its architecture or attention mechanisms?

Key findings

• RoBERTa's performance drops significantly under certain low-level attacks, with accuracy falling as low as the random guessing baseline (approximately 0.5) on the most severe attack types, such as visual and high-level phonetic perturbations.
• The visual attack mode caused the most dramatic performance degradation, reducing RoBERTa's accuracy to 0.70 on high-confidence inputs, indicating a critical vulnerability to visually similar character substitutions.
• Even after adversarial training with the Zéroe dataset, RoBERTa's performance improved only marginally, suggesting limited generalization of standard adversarial training to complex, human-plausible low-level attacks.
• The phonetic and natural noise attack types caused moderate but consistent performance drops, with accuracy falling to around 0.90–0.92 on high-confidence inputs, indicating that semantic and phonetic similarity alone is insufficient to preserve model robustness.
• The inner-shuffle and disemvowel attacks, while more perceptually plausible, caused less severe degradation, with accuracy remaining above 0.92, suggesting that RoBERTa retains some resilience to these specific perturbations.
• The benchmark reveals that RoBERTa is particularly sensitive to attacks that alter the visual or phonetic form of text without changing word boundaries, exposing a fundamental mismatch between human and model processing of orthographic variation.