[论文解读] Deep Speech Based End-to-End Automated Speech Recognition (ASR) for Indian-English Accents

Priyank Dubey, Bilal Shah|arXiv (Cornell University)|Apr 3, 2022

Speech Recognition and Synthesis被引用 14

一句话总结

本文提出了一种基于微调的端到端自动语音识别（ASR）系统，针对印度英语口音，采用迁移学习和数据增强技术，在Indic TTS数据集上进行训练。通过将预训练的DeepSpeech-0.9.3模型适配至印度英语语音，该方法在未经训练的基线模型和商业服务上均实现了显著性能提升，展现出在印度多样化区域英语口音上的强大表现。

ABSTRACT

Automated Speech Recognition (ASR) is an interdisciplinary application of computer science and linguistics that enable us to derive the transcription from the uttered speech waveform. It finds several applications in Military like High-performance fighter aircraft, helicopters, air-traffic controller. Other than military speech recognition is used in healthcare, persons with disabilities and many more. ASR has been an active research area. Several models and algorithms for speech to text (STT) have been proposed. One of the most recent is Mozilla Deep Speech, it is based on the Deep Speech research paper by Baidu. Deep Speech is a state-of-art speech recognition system is developed using end-to-end deep learning, it is trained using well-optimized Recurrent Neural Network (RNN) training system utilizing multiple Graphical Processing Units (GPUs). This training is mostly done using American-English accent datasets, which results in poor generalizability to other English accents. India is a land of vast diversity. This can even be seen in the speech, there are several English accents which vary from state to state. In this work, we have used transfer learning approach using most recent Deep Speech model i.e., deepspeech-0.9.3 to develop an end-to-end speech recognition system for Indian-English accents. This work utilizes fine-tuning and data argumentation to further optimize and improve the Deep Speech ASR system. Indic TTS data of Indian-English accents is used for transfer learning and fine-tuning the pre-trained Deep Speech model. A general comparison is made among the untrained model, our trained model and other available speech recognition services for Indian-English Accents.

研究动机与目标

• 为解决现有ASR系统在主要基于美式英语训练的情况下对印度英语口音泛化能力差的问题。
• 开发一种针对印度英语语音模式语言多样性的鲁棒端到端ASR系统。
• 利用预训练的DeepSpeech-0.9.3模型进行迁移学习，以减少对训练数据和计算资源的需求。
• 通过在Indic TTS数据集上对印度英语语音进行微调和数据增强，优化模型性能。

提出的方法

• 在包含印度英语口音的Indic TTS数据集上，使用迁移学习对预训练的DeepSpeech-0.9.3模型进行微调。
• 应用数据增强技术以增加训练数据的多样性，提升模型鲁棒性。
• 采用基于深度双向长短期记忆（Bi-LSTM）网络的序列到序列架构，并使用连接时序分类（CTC）损失函数。
• 使用多张GPU进行训练，以加速收敛并提高优化效率。
• 通过对比未训练模型、所提模型与商业ASR服务的词错误率（WER）来评估性能。

实验结果

研究问题

• RQ1 在领域特定数据有限的情况下，使用预训练的DeepSpeech模型进行迁移学习是否能有效适应印度英语口音？
• RQ2 数据增强对印度英语语音ASR模型的鲁棒性和准确性有何影响？
• RQ3 微调后的DeepSpeech模型在印度英语语音识别任务中相比未训练基线模型的性能提升如何？
• RQ4 所提出的系统在印度英语口音识别准确性方面与现有商业ASR服务相比表现如何？

主要发现

• 与未训练的基线模型相比，微调后的DeepSpeech模型在印度英语语音上实现了显著降低的词错误率（WER）。
• 所提出的系统在印度英语口音的WER表现上优于商业ASR服务，表明其在领域特定适应方面具有更强性能。
• 数据增强有助于提升模型泛化能力，尤其对印度境内稀有或低资源区域口音的识别效果改善明显。
• 迁移学习使预训练模型能够以极少的额外训练数据和计算成本实现有效适应。