[Paper Review] Isolated and Ensemble Audio Preprocessing Methods for Detecting Adversarial Examples against Automatic Speech Recognition

Krishan Rajaratnam, Kunal Shah+1 more|arXiv (Cornell University)|Sep 11, 2018

Digital Media Forensic Detection19 citations

TL;DR

This paper proposes isolated and ensemble audio preprocessing techniques to detect adversarial examples in automatic speech recognition systems. By combining compression, speech coding, filtering, and audio panning, the method achieves 93.5% precision and 91.2% recall in detecting adversarial inputs targeting the Speech Commands Model, demonstrating a robust defense against genetic attacks.

ABSTRACT

An adversarial attack is an exploitative process in which minute alterations are made to natural inputs, causing the inputs to be misclassified by neural models. In the field of speech recognition, this has become an issue of increasing significance. Although adversarial attacks were originally introduced in computer vision, they have since infiltrated the realm of speech recognition. In 2017, a genetic attack was shown to be quite potent against the Speech Commands Model. Limited-vocabulary speech classifiers, such as the Speech Commands Model, are used in a variety of applications, particularly in telephony; as such, adversarial examples produced by this attack pose as a major security threat. This paper explores various methods of detecting these adversarial examples with combinations of audio preprocessing. One particular combined defense incorporating compressions, speech coding, filtering, and audio panning was shown to be quite effective against the attack on the Speech Commands Model, detecting audio adversarial examples with 93.5% precision and 91.2% recall.

Motivation & Objective

• To address the growing threat of adversarial attacks on automatic speech recognition systems, particularly in limited-vocabulary models like the Speech Commands Model.
• To investigate whether audio preprocessing techniques can serve as effective detection mechanisms for adversarial examples.
• To evaluate both isolated and ensemble combinations of preprocessing methods for improved detection robustness.
• To develop a defense strategy that is computationally efficient and compatible with real-time speech recognition pipelines.

Proposed method

• The authors apply isolated preprocessing techniques including audio compression, speech coding (e.g., AMR), band-pass filtering, and stereo panning to input audio signals.
• They evaluate the effectiveness of each individual method in detecting adversarial examples generated via a genetic attack on the Speech Commands Model.
• An ensemble approach combines multiple preprocessing techniques—compression, speech coding, filtering, and panning—into a single detection pipeline.
• The detection system classifies inputs as benign or adversarial based on changes in model confidence scores after preprocessing.
• The method relies on the principle that adversarial perturbations are sensitive to signal transformations, while natural speech remains robust.
• The final detection model uses a threshold-based decision rule on the model’s output confidence following preprocessing.

Experimental results

Research questions

• RQ1 Can isolated audio preprocessing techniques effectively detect adversarial examples in automatic speech recognition systems?
• RQ2 How does the performance of individual preprocessing methods compare in detecting adversarial inputs?
• RQ3 Does combining multiple preprocessing techniques through an ensemble approach improve detection accuracy compared to isolated methods?
• RQ4 To what extent do adversarial examples remain effective after undergoing common audio signal transformations?
• RQ5 Can the proposed method maintain high detection performance while preserving the recognition accuracy of natural speech?

Key findings

• The ensemble method combining compression, speech coding, filtering, and audio panning achieved 93.5% precision and 91.2% recall in detecting adversarial examples.
• Individual preprocessing techniques showed varying levels of effectiveness, with speech coding and filtering demonstrating strong detection capabilities.
• The ensemble approach outperformed isolated methods, indicating synergistic benefits from combining multiple signal transformations.
• The method maintained high detection performance across multiple adversarial examples generated using a genetic attack.
• The defense mechanism preserved the recognition accuracy of natural speech, indicating minimal impact on legitimate inputs.