[Paper Review] Adversarial defense for automatic speaker verification by cascaded self-supervised learning models

Haibin Wu, Xu Li+4 more|arXiv (Cornell University)|Feb 14, 2021

Adversarial Robustness in Machine Learning30 references9 citations

TL;DR

This paper proposes a cascaded self-supervised learning defense using pretrained TERA models to purify adversarial perturbations in automatic speaker verification (ASV) systems. By placing multiple TERA models upstream of the ASV system, the method effectively mitigates both white-box and black-box adversarial attacks while preserving performance on genuine inputs, outperforming hand-crafted filters and defense methods requiring attack-specific knowledge.

ABSTRACT

Automatic speaker verification (ASV) is one of the core technologies in biometric identification. With the ubiquitous usage of ASV systems in safety-critical applications, more and more malicious attackers attempt to launch adversarial attacks at ASV systems. In the midst of the arms race between attack and defense in ASV, how to effectively improve the robustness of ASV against adversarial attacks remains an open question. We note that the self-supervised learning models possess the ability to mitigate superficial perturbations in the input after pretraining. Hence, with the goal of effective defense in ASV against adversarial attacks, we propose a standard and attack-agnostic method based on cascaded self-supervised learning models to purify the adversarial perturbations. Experimental results demonstrate that the proposed method achieves effective defense performance and can successfully counter adversarial attacks in scenarios where attackers may either be aware or unaware of the self-supervised learning models.

Motivation & Objective

• To address the growing threat of adversarial attacks on safety-critical automatic speaker verification (ASV) systems.
• To develop a defense method that is effective regardless of whether attackers know the defense mechanism, avoiding overfitting to specific attack patterns.
• To improve ASV robustness without modifying the internal architecture of existing ASV models.
• To explore the feasibility of using self-supervised models as deep, attack-agnostic filters to remove adversarial noise.
• To evaluate the defense under both unaware and aware attacker scenarios, including transferability and white-box attack resistance.

Proposed method

• The defense employs cascaded TERA (Transformer Encoder Representations from Alteration) models as upstream feature extractors to purify adversarial inputs before they reach the ASV system.
• TERA models are pretrained on a denoising autoencoding task, enabling them to reconstruct clean speech from noisy or perturbed inputs, including adversarial perturbations.
• The method is applied as a plug-in filter: multiple TERA models are stacked in series to progressively remove adversarial noise, forming a cascaded purification pipeline.
• The defense is attack-agnostic, requiring no knowledge of the attack's specific parameters or generation strategy.
• The approach is evaluated in two scenarios: attackers unaware of TERA (black-box) and attackers aware of TERA (white-box), with adversarial samples generated via BIM with ε=0.3.
• Performance is measured using equal error rate (EER) on both genuine and adversarial inputs, comparing against hand-crafted filters (Gaussian, median, mean).

Experimental results

Research questions

• RQ1 Can cascaded TERA models effectively reduce the success rate of adversarial attacks on ASV systems in a black-box setting?
• RQ2 How does the defense perform when attackers are aware of the TERA models and can craft targeted attacks?
• RQ3 Does the cascaded TERA defense preserve the performance of ASV on genuine, unperturbed speech inputs?
• RQ4 How does the cascaded TERA defense compare to traditional hand-crafted filters (e.g., Gaussian, median) in both adversarial and clean input scenarios?
• RQ5 Can increasing the number of TERA models in cascade further improve defense robustness against white-box attacks?

Key findings

• The cascaded TERA defense reduced the EER on adversarial inputs from 66.02% (no defense) to 22.94%, demonstrating strong mitigation of attack effectiveness.
• Even when attackers are aware of the TERA models and use them to generate adversarial samples, increasing the number of TERA models in cascade still reduces EER from 54.55% (no TERA) to 40.69% (three TERA models).
• The TERA-based defense outperformed hand-crafted filters: it achieved lower EER on adversarial inputs (22.94% vs. 29.44–31.60%) and better preserved performance on genuine inputs (17.32% vs. 27.06–30.30%).
• The method maintained robustness in both black-box and white-box attack scenarios, showing it is effective even when attackers have full knowledge of the defense mechanism.
• The performance degradation on genuine inputs (EER increased from 8.87% to 17.32%) was less severe than with hand-crafted filters, indicating better reconstruction fidelity of TERA models.
• The defense is computationally intensive, especially in white-box settings, but the cascaded design provides a scalable defense strategy against adaptive attackers.