[Paper Review] AlignTTS: Efficient Feed-Forward Text-to-Speech System without Explicit Alignment

Zhen Zeng, Jianzong Wang+3 more|arXiv (Cornell University)|Mar 4, 2020

Speech Recognition and Synthesis27 references4 citations

TL;DR

AlignTTS proposes a non-autoregressive, feed-forward Transformer-based text-to-speech system that achieves state-of-the-art performance without relying on autoregressive teacher models. By introducing a novel alignment loss inspired by the Baum-Welch algorithm, it learns precise text-to-mel-spectrum alignment during training, enabling parallel inference that is over 50× faster than real-time while improving MOS by 0.03 over Transformer TTS.

ABSTRACT

Targeting at both high efficiency and performance, we propose AlignTTS to predict the mel-spectrum in parallel. AlignTTS is based on a Feed-Forward Transformer which generates mel-spectrum from a sequence of characters, and the duration of each character is determined by a duration predictor.Instead of adopting the attention mechanism in Transformer TTS to align text to mel-spectrum, the alignment loss is presented to consider all possible alignments in training by use of dynamic programming. Experiments on the LJSpeech dataset show that our model achieves not only state-of-the-art performance which outperforms Transformer TTS by 0.03 in mean option score (MOS), but also a high efficiency which is more than 50 times faster than real-time.

Motivation & Objective

• To develop a high-efficiency, non-autoregressive text-to-speech system that avoids reliance on autoregressive teacher models.
• To improve alignment learning between text and mel-spectrum without explicit attention mechanisms.
• To achieve state-of-the-art speech quality while enabling real-time inference speed.
• To design a training objective that considers all possible alignments, improving robustness and accuracy.

Proposed method

• The model uses a feed-forward Transformer (FFT) with stacked FFT blocks, self-attention, and 1D convolutions to generate mel-spectrograms in parallel.
• A duration predictor estimates the pronunciation duration of each character, which is used by a length regulator to align text and mel-spectrum during inference.
• An alignment loss is introduced, inspired by the Baum-Welch algorithm, to train the model by considering all possible alignments during backpropagation.
• A mix density network is used to model the conditional distribution of mel-spectrograms given text, improving alignment learning.
• The model is trained end-to-end using a combination of reconstruction loss, duration loss, and alignment loss.
• WaveGlow is used as the parallel neural vocoder to synthesize waveforms from predicted mel-spectrograms.

Experimental results

Research questions

• RQ1 Can a non-autoregressive TTS system achieve state-of-the-art performance without using a pre-trained autoregressive model for alignment guidance?
• RQ2 Can a differentiable alignment loss that considers all possible alignments improve alignment accuracy and synthesis quality?
• RQ3 Can a feed-forward Transformer architecture enable high-speed inference while maintaining high speech quality?
• RQ4 How does the proposed alignment loss compare to attention-based alignment in terms of precision and robustness?

Key findings

• AlignTTS achieves a mean opinion score (MOS) of 4.05±0.12 on the LJSpeech dataset, outperforming Transformer TTS by 0.03 MOS.
• The model synthesizes approximately 10 seconds of speech in 0.18 seconds, achieving over 50× speedup compared to real-time.
• The alignment loss enables more precise alignment between text and mel-spectrum than the attention mechanism in Transformer TTS, as visually confirmed in Figure 3.
• The inference speed of AlignTTS is 0.18 seconds per 10 seconds of speech, with 0.06 seconds for the TTS model and 0.12 seconds for WaveGlow vocoder.
• The model achieves a significant speedup over autoregressive models like Tacotron2 (4.58s) and Transformer TTS (3.26s) while maintaining higher quality.
• The duration predictor and length regulator allow control over speech rate and pauses, enabling prosodic control in inference.