[Paper Review] Hybrid CTC-Attention based End-to-End Speech Recognition using Subword Units

Zhangyu Xiao, Zhijian Ou+2 more|arXiv (Cornell University)|Jul 13, 2018

Speech Recognition and Synthesis21 references4 citations

TL;DR

This paper proposes a hybrid CTC-Attention end-to-end speech recognition system using byte-pair encoding (BPE)-derived subword units, achieving 6.8% WER on LibriSpeech's test_clean set without a dictionary or external language model. The approach improves over character-based systems by reducing WER by 12.8% relative, leveraging subwords to mitigate out-of-vocabulary issues and model longer context more effectively.

ABSTRACT

In this paper, we present an end-to-end automatic speech recognition system, which successfully employs subword units in a hybrid CTC-Attention based system. The subword units are obtained by the byte-pair encoding (BPE) compression algorithm. Compared to using words as modeling units, using characters or subword units does not suffer from the out-of-vocabulary (OOV) problem. Furthermore, using subword units further offers a capability in modeling longer context than using characters. We evaluate different systems over the LibriSpeech 1000h dataset. The subword-based hybrid CTC-Attention system obtains 6.8% word error rate (WER) on the test_clean subset without any dictionary or external language model. This represents a significant improvement (a 12.8% WER relative reduction) over the character-based hybrid CTC-Attention system.

Motivation & Objective

• To address the limitations of character-based end-to-end ASR, such as poor modeling of long-range dependencies and high data sparsity.
• To overcome the out-of-vocabulary (OOV) problem inherent in word- and phoneme-based systems by using subword units.
• To explore the effectiveness of combining CTC and attention losses in a hybrid architecture using subword units.
• To evaluate the impact of subword unit size and training data scale on system performance.

Proposed method

• Subword units are generated using the byte-pair encoding (BPE) algorithm on the full training transcript corpus.
• A hybrid CTC-Attention neural network architecture is employed, combining CTC loss for monotonic alignment and attention-based decoder loss for context-aware generation.
• The model uses an 8-layer bidirectional LSTM encoder with batch normalization and a unidirectional LSTM decoder with location-based attention.
• Training is optimized using Adadelta with gradient clipping and a weighted combination of CTC and attention losses, with hyperparameter λ tuned to 0.2.
• Beam search decoding with beam size 20 is used during inference, and the system is trained on 1000h LibriSpeech data using 4 Tesla K80 GPUs.
• The subword unit vocabulary size is set to 500, and the model is evaluated without any external language model or lexicon.

Experimental results

Research questions

• RQ1 Can subword units improve end-to-end speech recognition performance in a hybrid CTC-Attention framework compared to character-level modeling?
• RQ2 How does the use of subword units affect the out-of-vocabulary (OOV) problem and long-context modeling in end-to-end ASR?
• RQ3 What is the optimal number of subword units for achieving low WER on LibriSpeech without external language models?
• RQ4 How does training data size influence the performance of subword-based end-to-end ASR systems?

Key findings

• The subword-based hybrid CTC-Attention system achieves 6.8% WER on the LibriSpeech test_clean set without any dictionary or external language model.
• This represents a 12.8% relative reduction in WER compared to the character-based hybrid system, demonstrating significant performance gain.
• Using 1000 subword units yields slightly better performance than character-based models, but performance degrades with larger subword vocabularies (e.g., 1000 units) due to data sparsity.
• The model trained on 100h of data achieves 34.7% WER on test_clean, while performance improves to 10.4% on 500h and 6.8% on 1000h, indicating strong data scaling benefits.
• The optimal λ value for the hybrid loss is 0.2, and increasing λ to 0.5 degrades performance due to CTC's poor performance without language modeling.
• Subword units enable open-vocabulary recognition and improve modeling of rare or compound words, reducing OOV issues compared to word-level units.