[论文解读] Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends

Siddique Latif, Rajib Rana+4 more|arXiv (Cornell University)|Jan 2, 2020

Speech Recognition and Synthesis参考文献 363被引用 67

一句话总结

对跨 ASR、SR 和 SER 的语音深度表征学习的全面综述，涵盖模型、技术、挑战和未来方向。

ABSTRACT

Research on speech processing has traditionally considered the task of designing hand-engineered acoustic features (feature engineering) as a separate distinct problem from the task of designing efficient machine learning (ML) models to make prediction and classification decisions. There are two main drawbacks to this approach: firstly, the feature engineering being manual is cumbersome and requires human knowledge; and secondly, the designed features might not be best for the objective at hand. This has motivated the adoption of a recent trend in speech community towards utilisation of representation learning techniques, which can learn an intermediate representation of the input signal automatically that better suits the task at hand and hence lead to improved performance. The significance of representation learning has increased with advances in deep learning (DL), where the representations are more useful and less dependent on human knowledge, making it very conducive for tasks like classification, prediction, etc. The main contribution of this paper is to present an up-to-date and comprehensive survey on different techniques of speech representation learning by bringing together the scattered research across three distinct research areas including Automatic Speech Recognition (ASR), Speaker Recognition (SR), and Speaker Emotion Recognition (SER). Recent reviews in speech have been conducted for ASR, SR, and SER, however, none of these has focused on the representation learning from speech -- a gap that our survey aims to bridge.

研究动机与目标

• 缩小语音领域表示学习研究的碎片化差异，并提供一个涵盖 ASR、SR、SER 的最新综述。
• 总结用于语音处理的深度学习模型和表示学习技术。
• 讨论深度语音表征的挑战、关键特征及最新进展。
• 强调数据集、评估指标以及未来趋势以指导研究者。

提出的方法

• 回顾语音中的传统特征学习与深度特征学习的对比，以及从手工特征到自动学习表征的转变。
• 总结用于语音表征学习的深度学习架构（DNNs、CNNs、RNNs、AEs、VAEs、GANs）及它们的作用。
• 讨论深度表示学习在 ASR、SR 和 SER 中的应用及相关训练范式（有监督、无监督、迁移、强化）。
• 概述语音表示学习研究中常用的数据集和评估指标。
• 突出噪声鲁棒性、数据需求和泛化等挑战，并指出未来的研究方向。

实验结果

研究问题

• RQ1 在语音处理中应用的主要深度表示学习技术有哪些？
• RQ2 表示学习方法在 ASR、说话人识别和情感识别领域的表现如何？
• RQ3 哪些挑战和未来趋势将塑造语音中的深度表示学习？

主要发现

• 本文提供了在三个核心语音领域：ASR、SR 和 SER 的最新表示学习技术综述。
• 它涵盖了包括 DNNs、CNNs、RNNs、AEs、VAEs、GANs 以及深度自回归模型在内的深度模型与表征。
• 它讨论了应用场景、挑战和最新进展，以及领域中使用的数据集和评估指标。
• 它强调了从手工特征工程向自动表示学习的转变，以及数据可用性和模型架构的重要性。
• 它概述了语音研究中深度表示学习的未来趋势与方向。