[Paper Review] Did you hear that? Adversarial Examples Against Automatic Speech Recognition

Moustafa Alzantot, Bharathan Balaji+1 more|arXiv (Cornell University)|Jan 2, 2018

Adversarial Robustness in Machine Learning11 references197 citations

TL;DR

The paper demonstrates a black-box, gradient-free genetic algorithm to generate targeted adversarial audio against a speech Command Recognition model, achieving 87% success while adding minimal, often imperceptible noise. Human listeners largely cannot distinguish the perturbations.

ABSTRACT

Speech is a common and effective way of communication between humans, and modern consumer devices such as smartphones and home hubs are equipped with deep learning based accurate automatic speech recognition to enable natural interaction between humans and machines. Recently, researchers have demonstrated powerful attacks against machine learning models that can fool them to produceincorrect results. However, nearly all previous research in adversarial attacks has focused on image recognition and object detection models. In this short paper, we present a first of its kind demonstration of adversarial attacks against speech classification model. Our algorithm performs targeted attacks with 87% success by adding small background noise without having to know the underlying model parameter and architecture. Our attack only changes the least significant bits of a subset of audio clip samples, and the noise does not change 89% the human listener's perception of the audio clip as evaluated in our human study.

Motivation & Objective

• Motivate and demonstrate adversarial vulnerabilities in ASR systems beyond image models.
• Show that black-box, gradient-free attacks can target specific outputs in speech recognition.
• Quantify the perceptual impact of adversarial noise on human listeners through a user study.
• Provide a reproducible methodology and publicly available implementation for generating adversarial audio.

Proposed method

• Use a genetic algorithm to generate targeted adversarial audio without requiring model gradients or architecture details.
• Perturb only the least-significant bits of a subset of audio samples to minimize perceptual impact.
• Fitness evaluates the likelihood of the target label under the black-box model and guides selection, crossover, and mutation.
• Limit attacks to 500 iterations with a 37-second median generation time on a Titan X GPU.
• Evaluate on the Speech Commands model and dataset with 500 source clips and 9 target labels per clip (4500 adversarial examples).

Experimental results

Research questions

• RQ1 Can a black-box, gradient-free attack substantially mislead ASR into a chosen target label?
• RQ2 What is the success rate of targeted adversarial audio against a practical ASR model?
• RQ3 How perceptible are the perturbations to human listeners, and do humans retain the original label perception?
• RQ4 Is the approach reproducible with publicly available code and parameters?
• RQ5 How do untargeted attacks compare in feasibility and impact to targeted attacks in ASR?

Key findings

• Targeted attacks succeed in 87% of cases across source-target pairs.
• Noise is added only to the 8 least-significant bits of a subset of samples in a 16-bit audio file.
• Human study (23 participants, 1500 clips) found 89% of the time the noise did not change human labeling from the source label.
• Median adversarial example generation time is 37 seconds on a Titan X GPU.
• No MFCC gradients are required; the attack is demonstrated in a black-box setting, avoiding model internals.
• The attack achieves 100% success for untargeted attacks in their discussion (not in the main results).