[Paper Review] Adversarial Defense for Automatic Speaker Verification by Self-Supervised Learning

Haibin Wu, Xu Li+4 more|arXiv (Cornell University)|Jun 1, 2021

Adversarial Robustness in Machine Learning58 references4 citations

TL;DR

This paper proposes a self-supervised learning (SSL)-based defense for automatic speaker verification (ASV) that operates without prior knowledge of adversarial attack methods. By treating adversarial perturbations as noise, it employs SSL models for both perturbation purification and detection, achieving ~80% accuracy in detecting adversarial samples and introducing a formal evaluation framework for ASV adversarial defense.

ABSTRACT

Previous works have shown that automatic speaker verification (ASV) is seriously vulnerable to malicious spoofing attacks, such as replay, synthetic speech, and recently emerged adversarial attacks. Great efforts have been dedicated to defending ASV against replay and synthetic speech; however, only a few approaches have been explored to deal with adversarial attacks. All the existing approaches to tackle adversarial attacks for ASV require the knowledge for adversarial samples generation, but it is impractical for defenders to know the exact attack algorithms that are applied by the in-the-wild attackers. This work is among the first to perform adversarial defense for ASV without knowing the specific attack algorithms. Inspired by self-supervised learning models (SSLMs) that possess the merits of alleviating the superficial noise in the inputs and reconstructing clean samples from the interrupted ones, this work regards adversarial perturbations as one kind of noise and conducts adversarial defense for ASV by SSLMs. Specifically, we propose to perform adversarial defense from two perspectives: 1) adversarial perturbation purification and 2) adversarial perturbation detection. Experimental results show that our detection module effectively shields the ASV by detecting adversarial samples with an accuracy of around 80%. Moreover, since there is no common metric for evaluating the adversarial defense performance for ASV, this work also formalizes evaluation metrics for adversarial defense considering both purification and detection based approaches into account. We sincerely encourage future works to benchmark their approaches based on the proposed evaluation framework.

Motivation & Objective

• Address the critical vulnerability of automatic speaker verification (ASV) to adversarial attacks, which remain under-explored compared to replay and synthetic speech attacks.
• Overcome the limitation of existing defenses that require prior knowledge of attack algorithms, which is impractical in real-world scenarios.
• Develop a defense mechanism for ASV that is robust against unknown or in-the-wild adversarial attacks by leveraging the noise-robustness of self-supervised learning models (SSLMs).
• Formalize evaluation metrics for adversarial defense in ASV, covering both purification and detection-based approaches, to enable standardized benchmarking.

Proposed method

• Treat adversarial perturbations as a form of input noise and leverage the inherent noise-reduction capability of self-supervised learning models (SSLMs) for adversarial perturbation purification.
• Design a detection module that identifies adversarial samples by analyzing discrepancies between original and reconstructed features via SSLMs.
• Utilize pre-trained SSLMs (e.g., Wav2Vec 2.0 or similar) to extract robust representations that are less sensitive to adversarial perturbations.
• Train a classifier on SSL features to distinguish between clean and adversarial inputs, enabling real-time detection of malicious samples.
• Integrate both purification and detection modules into a unified defense pipeline to enhance ASV robustness.
• Propose a formal evaluation framework that accounts for both purification and detection performance, including metrics like detection accuracy and verification accuracy under attack.

Experimental results

Research questions

• RQ1 Can self-supervised learning models effectively purify adversarial perturbations in ASV inputs without prior knowledge of the attack method?
• RQ2 How effective is a detection-based defense using SSLMs in identifying adversarial samples in ASV systems?
• RQ3 What evaluation metrics are most appropriate for benchmarking adversarial defense in ASV, considering both purification and detection components?
• RQ4 Can a defense mechanism that does not rely on attack-specific knowledge still achieve high robustness against diverse adversarial attacks?

Key findings

• The proposed detection module achieves approximately 80% accuracy in identifying adversarial samples, demonstrating strong performance without requiring knowledge of the attack algorithm.
• The SSL-based purification approach effectively reduces the impact of adversarial perturbations by leveraging the noise-robustness of self-supervised representations.
• The integration of both purification and detection components leads to a more resilient ASV system under adversarial conditions.
• The paper formalizes a comprehensive evaluation framework for adversarial defense in ASV, addressing the lack of standardized metrics in prior work.
• Experimental results confirm that the defense mechanism is effective even when the attack method is unknown, highlighting its practical applicability.
• The proposed evaluation framework is expected to enable fair and consistent benchmarking of future adversarial defense methods in ASV.