[论文解读] Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning

Haibin Wu, Xu Li+4 more|arXiv (Cornell University)|Jun 1, 2021

Speech Recognition and Synthesis被引用 5

一句话总结

本文提出一种基于自监督学习（SSL）的防御方法，用于自动说话人验证（ASV）系统，以抵御对抗性攻击，且无需事先了解攻击方法。该方法采用双重策略：利用自监督重构网络（self-supervised reformer）净化对抗性扰动，以恢复干净语音；并通过统计分析多个SSL模型生成的ASV评分，检测对抗性样本。该方法实现了79.0%–83.3%的检测准确率，并将对抗性误报率（FAR）从超过67%降低至30%以下。

ABSTRACT

Previous works have shown that automatic speaker verification (ASV) is seriously vulnerable to malicious spoofing attacks, such as replay, synthetic speech, and recently emerged adversarial attacks. Great efforts have been dedicated to defending ASV against replay and synthetic speech; however, only a few approaches have been explored to deal with adversarial attacks. All the existing approaches to tackle adversarial attacks for ASV require the knowledge for adversarial samples generation, but it is impractical for defenders to know the exact attack algorithms that are applied by the in-the-wild attackers. This work is among the first to perform adversarial defense for ASV without knowing the specific attack algorithms. Inspired by self-supervised learning models (SSLMs) that possess the merits of alleviating the superficial noise in the inputs and reconstructing clean samples from the interrupted ones, this work regards adversarial perturbations as one kind of noise and conducts adversarial defense for ASV by SSLMs. Specifically, we propose to perform adversarial defense from two perspectives: 1) adversarial perturbation purification and 2) adversarial perturbation detection. Experimental results show that our detection module effectively shields the ASV by detecting adversarial samples with an accuracy of around 80%. Moreover, since there is no common metric for evaluating the adversarial defense performance for ASV, this work also formalizes evaluation metrics for adversarial defense considering both purification and detection based approaches into account. We sincerely encourage future works to benchmark their approaches based on the proposed evaluation framework.

研究动机与目标

• 解决说话人验证系统在对抗性攻击面前的严重脆弱性问题，该问题相较于重放攻击和合成语音攻击仍研究不足。
• 开发一种不依赖于攻击者所用具体对抗性攻击算法知识的ASV防御机制。
• 利用自监督学习模型（SSLM）对噪声的鲁棒性，实现对抗性扰动的净化与恶意输入的检测。
• 建立标准化的ASV对抗性防御评估框架，涵盖净化与检测两类方法的评价指标。
• 在多种攻击类型（FGSM、BIM、JSMA）和ASV系统（r-vector、x-vector）上验证该防御方法的泛化能力。

提出的方法

• 使用自监督重构网络（SSLR）作为净化模块，将对抗性扰动输入视为噪声，重建出干净语音。
• 在对抗性样本上微调SSLR，以增强其恢复真实说话人特征的能力，同时最小化对干净样本的性能退化。
• 设计检测模块，分析将ASV系统与不同数量级级联的SSLM集成后生成的ASV评分的统计分布。
• 从评分分布中提取高阶统计矩（最高至五阶），以检测指示对抗性输入的异常特征。
• 将检测模块与ASV模型集成于串联系统中，在验证前过滤掉对抗性样本。
• 提出一种结合净化与检测指标的新评估框架，包含AdvFAR、AdvFRR、j-FAR、j-FRR和EER_det等指标，以实现公平基准测试。

实验结果

研究问题

• RQ1 在未知攻击方法的前提下，自监督学习模型能否有效净化语音验证输入中的对抗性扰动？
• RQ2 通过集成多个SSLM后ASV评分的统计差异，能否可靠检测出对抗性样本？
• RQ3 与传统滤波基线相比，所提出的净化方法在对抗性攻击下对真实说话人验证性能的保持能力如何？
• RQ4 该检测模块在不同对抗性攻击类型（如FGSM、BIM、JSMA）之间具有多大程度的泛化能力？
• RQ5 在语音验证对抗性防御中，哪些评价指标最为合适？如何实现其标准化？

主要发现

• 基于SSLR的净化模块将r-vector系统的对抗性FAR从72.30%降低至20.82%，x-vector系统则从67.29%降低至26.21%，显著提升了系统鲁棒性。
• 该方法在真实样本上保持了高精度，微调后r-vector系统的EER绝对下降仅3.12%，x-vector系统为1.70%。
• 检测模块对r-vector系统在BIM攻击下的检测准确率达79.0%，x-vector系统达83.3%，JSMA攻击下EER_det最低可达13.6%。
• 检测模块在不同攻击类型间表现出良好泛化能力，EER_det范围为13.6%（JSMA）至31.6%（FGSM），表明更有效的攻击更具可检测性。
• 将检测模块与ASV系统集成后，r-vector系统的j-FAR从39.08%降至10.01%，x-vector系统从31.21%降至9.17%，尽管j-FRR略有上升，但实际影响显著。
• 所提出的评估框架（包含AdvFAR、j-FAR、j-FRR、EER_det等指标）为未来ASV对抗性防御研究提供了标准化基准。