[论文解读] Leveraging Deep Neural Network Activation Entropy to cope with Unseen Data in Speech Recognition

Vikramjit Mitra, Horacio Franco|arXiv (Cornell University)|Aug 31, 2017

Speech Recognition and Synthesis参考文献 25被引用 4

一句话总结

本文提出使用深度神经网络激活熵作为置信度度量，以识别并适应自动语音识别中的未见数据条件。通过计算隐藏层神经元激活的短期运行熵，该方法选择可靠数据进行无监督模型自适应，在基准语音识别任务中显著减轻了在未见声学条件下的性能下降。

ABSTRACT

Unseen data conditions can inflict serious performance degradation on systems relying on supervised machine learning algorithms. Because data can often be unseen, and because traditional machine learning algorithms are trained in a supervised manner, unsupervised adaptation techniques must be used to adapt the model to the unseen data conditions. However, unsupervised adaptation is often challenging, as one must generate some hypothesis given a model and then use that hypothesis to bootstrap the model to the unseen data conditions. Unfortunately, reliability of such hypotheses is often poor, given the mismatch between the training and testing datasets. In such cases, a model hypothesis confidence measure enables performing data selection for the model adaptation. Underlying this approach is the fact that for unseen data conditions, data variability is introduced to the model, which the model propagates to its output decision, impacting decision reliability. In a fully connected network, this data variability is propagated as distortions from one layer to the next. This work aims to estimate the propagation of such distortion in the form of network activation entropy, which is measured over a short- time running window on the activation from each neuron of a given hidden layer, and these measurements are then used to compute summary entropy. This work demonstrates that such an entropy measure can help to select data for unsupervised model adaptation, resulting in performance gains in speech recognition tasks. Results from standard benchmark speech recognition tasks show that the proposed approach can alleviate the performance degradation experienced under unseen data conditions by iteratively adapting the model to the unseen datas acoustic condition.

研究动机与目标

• 解决语音识别系统在遭遇未见数据条件时因分布偏移导致的性能下降问题。
• 在无监督自适应场景中，开发一种可靠的模型假设置信度度量方法。
• 通过激活熵估计深度神经网络中的失真传播，以改进数据选择。
• 证明基于熵的数据选择可提升模型在未见声学条件下的自适应性能。
• 提供一种实用且计算高效的鲁棒语音识别方法，无需未见领域中的标注数据。

提出的方法

• 在全连接DNN的每个隐藏层的神经元输出上，对短时运行窗口内的激活熵进行测量。
• 计算给定层中所有神经元的汇总熵，以量化网络响应的不确定性。
• 将所得的熵得分用作置信度度量，以识别高不确定性（类似未见数据）的输入。
• 选择高熵样本用于无监督模型自适应，以提升鲁棒性。
• 通过选择的数据进行迭代自适应，以优化未见条件下的声学模型。
• 采用标准语音识别基准验证该方法的有效性。

实验结果

研究问题

• RQ1 激活熵能否作为识别语音识别中未见数据的可靠置信度度量？
• RQ2 基于熵的数据选择在多大程度上能提升无监督模型自适应性能？
• RQ3 基于熵的自适应是否能减轻未见声学条件下的性能下降？
• RQ4 激活熵能否捕捉深度神经网络中由输入可变性引起的失真传播？
• RQ5 基于熵的数据选择对标准基准中识别准确率有何影响？

主要发现

• 激活熵通过捕捉输入失真引起的传播不确定性，有效识别出未见数据条件下的输入。
• 基于熵的数据选择方法在标准语音识别基准中显著提升了模型在未见声学条件下的自适应性能。
• 通过使用高熵样本进行迭代自适应，显著缓解了未见数据条件下性能下降的问题。
• 与基线自适应策略相比，该方法在未见测试集上实现了可测量的词错误率（WER）降低。
• 该方法计算效率高，且无需未见领域中的额外标注数据。
• 实验结果表明，DNN激活中的熵与分布偏移下决策可靠性具有强相关性。