[论文解读] VENOMAVE: Clean-Label Poisoning Against Speech Recognition.

Hojjat Aghakhani, Thorsten Eisenhofer+5 more|arXiv (Cornell University)|Oct 21, 2020

Adversarial Robustness in Machine Learning参考文献 3被引用 11

一句话总结

VENOMAVE 是首个针对自动语音识别（ASR）系统的数据中毒攻击，利用干净标签中毒技术，通过在训练过程中注入经过细微修改的音频样本，操控 ASR 模型。通过在目标语音的时序帧上精心设计扰动，该攻击在仅 0.94% 的训练数据被污染的情况下，实现了 83.33% 的攻击成功率，揭示了 ASR 系统中的关键安全漏洞。

ABSTRACT

In the past few years, we observed a wide adoption of practical systems that use Automatic Speech Recognition (ASR) systems to improve human-machine interaction. Modern ASR systems are based on neural networks and prior research demonstrated that these systems are susceptible to adversarial examples, i.e., malicious audio inputs that lead to misclassification by the victim's network during the system's run time. The research question if ASR systems are also vulnerable to data poisoning attacks is still unanswered. In such an attack, a manipulation happens during the training phase of the neural network: an adversary injects malicious inputs into the training set such that the neural network's integrity and performance are compromised. In this paper, we present the first data poisoning attack in the audio domain, called VENOMAVE. Prior work in the image domain demonstrated several types of data poisoning attacks, but they cannot be applied to the audio domain. The main challenge is that we need to attack a time series of inputs. To enforce a targeted misclassification in an ASR system, we need to carefully generate a specific sequence of disturbed inputs for the target utterance, which will eventually be decoded to the desired sequence of words. More specifically, the adversarial goal is to produce a series of misclassification tasks and in each of them, we need to poison the system to misrecognize each frame of the target file. To demonstrate the practical feasibility of our attack, we evaluate VENOMAVE on an ASR system that detects sequences of digits from 0 to 9. When poisoning only 0.94% of the dataset on average, we achieve an attack success rate of 83.33%. We conclude that data poisoning attacks against ASR systems represent a real threat that needs to be considered.

研究动机与目标

• 调查 ASR 系统在模型训练期间是否易受数据中毒攻击的影响。
• 开发一种专为音频输入的时间序列结构量身定制的新颖数据中毒技术，区别于以往基于图像的方法。
• 证明以极低比例的数据注入实现 ASR 模型中毒的可行性，同时保持干净标签特性。
• 在真实世界的 ASR 任务中评估该攻击的有效性——即从 0 到 9 的数字序列识别。

提出的方法

• 设计一种时序感知的中毒策略，通过在目标音频语句的多个帧上生成扰动，诱导目标误分类。
• 优化扰动以保持不可察觉性并维持干净标签特性，确保中毒样本在训练过程中不易被检测到。
• 将攻击形式化为一个序列到序列的误分类任务，其中目标音频的每一帧都被污染，以引导 ASR 解码器输出期望的目标结果。
• 在包含中毒样本的数据集上训练 ASR 模型，确保模型学会将目标语句错误识别为期望的标签。
• 采用基于梯度的优化方法生成对抗性扰动，以最大化目标转录结果的可能性，同时最小化感知失真。

实验结果

研究问题

• RQ1 数据 poisoning 攻击能否有效应用于处理序列化音频输入的自动语音识别系统？
• RQ2 如何设计一种干净标签 poisoning 攻击，使其在保持不可察觉性的同时，诱导 ASR 模型产生目标误分类？
• RQ3 实现高成功率的 ASR 数据 poisoning 攻击所需的最小污染比例是多少？
• RQ4 音频输入的时间序列结构如何影响 data poisoning 攻击的设计与有效性？

主要发现

• VENOMAVE 在训练用于识别 0 到 9 数字序列的 ASR 系统上，成功实现了 83.33% 的攻击成功率。
• 该攻击平均仅需污染 0.94% 的训练数据集即可达到上述成功率。
• 该方法保持了干净标签特性，使中毒样本在训练过程中看起来合法。
• 尽管音频具有序列特性，带来了图像 poisoning 攻击中不存在的挑战，该攻击依然有效。
• 结果表明，音频领域的数据 poisoning 是一种切实可行且严重的 ASR 系统完整性威胁。