[論文レビュー] Deep Voice: Real-time Neural Text-to-Speech

Sercan Ö. Arık, Mike Chrzanowski+10 more|arXiv (Cornell University)|Feb 25, 2017

Speech Recognition and Synthesis参考文献 20被引用数 395

ひとこと要約

この論文は Deep Voice を提示します、完全なニューラルで生産品質の TTS システムで、five components（G2P、 segmentation、 duration、 F0、 and audio synthesis）を備え、大規模データセットでの WaveNet 推論と訓練をリアルタイムより速い実行を実証します。

ABSTRACT

We present Deep Voice, a production-quality text-to-speech system constructed entirely from deep neural networks. Deep Voice lays the groundwork for truly end-to-end neural speech synthesis. The system comprises five major building blocks: a segmentation model for locating phoneme boundaries, a grapheme-to-phoneme conversion model, a phoneme duration prediction model, a fundamental frequency prediction model, and an audio synthesis model. For the segmentation model, we propose a novel way of performing phoneme boundary detection with deep neural networks using connectionist temporal classification (CTC) loss. For the audio synthesis model, we implement a variant of WaveNet that requires fewer parameters and trains faster than the original. By using a neural network for each component, our system is simpler and more flexible than traditional text-to-speech systems, where each component requires laborious feature engineering and extensive domain expertise. Finally, we show that inference with our system can be performed faster than real time and describe optimized WaveNet inference kernels on both CPU and GPU that achieve up to 400x speedups over existing implementations.

研究の動機と目的

• 従来の手作業で設計された TTS パイプラインを完全なニューラルのエンドツーエンドシステムに置換する。
• 最適化された WaveNet ベースの音声合成を通じてリアルタイム対応の推論を実証する。
• 最小限の手動アノテーションとハイパーパラメータ調整で新しいデータセットへの適応性を示す。
• CPUとGPUハードウェア全体で効率的な訓練と推論パイプラインを開発する。

提案手法

• Five-block TTS architecture: grapheme-to-phoneme, segmentation (CTC-based), phoneme duration, F0, and audio synthesis."
• Grapheme-to-phoneme model based on a multi-layer bidirectional encoder with GRU decoder and beam search."
• Segmentation model uses convolutional recurrent network with CTC loss and phoneme-pair labeling to improve boundary detection."
• Joint phoneme duration and F0 model predicting duration, voicing, and 20 time-dependent F0 values with a combined loss."
• Audio synthesis model is a variant of WaveNet using a QRNN pre-encoder to improve training speed and a real-time capable, optimized inference pipeline; CPU and GPU kernels described for real-time performance.

実験結果

リサーチクエスチョン

• RQ1 Can a production-quality TTS system be built entirely from neural components without hand-engineered features?
• RQ2 Is real-time or faster-than-real-time audio synthesis achievable with optimized WaveNet-based vocoders?
• RQ3 How do the segmentation, duration, and F0 prediction components affect overall speech naturalness and intelligibility?
• RQ4 How does the system perform across different datasets (internal English, Blizzard Blizzard 2013) and what are the perceptual outcomes?
• RQ5 What are the trade-offs between model size, inference speed, and audio quality in a production setting?

主な発見

• Phoneme boundary detection via CTC-based segmentation yields a 7% phoneme-pair error rate after ~14k iterations.
• Grapheme-to-phoneme model attains phoneme error rate of 5.8% and word error rate of 28.7% on CMUDict-style data without a language model.
• Mean absolute error for phoneme duration is 38 ms and F0 error is 29.4 Hz after ~20k iterations.
• 40-layer WaveNet with QRNN pre-encoder produces high-quality speech; 20/30/40 layer models yield usable audio, with 40 layers offering less noise."
• MOS results show ground-truth 48 kHz speech around 4.75±0.12; synthesized 40-layer model with 48 kHz companded audio around 3.84±0.24; synthesized with ground-truth durations/F0 around 2.00±0.23; real-time CPU/GPU inference benchmarks demonstrate real-time and near real-time performance depending on model size.