[Paper Review] DiDiSpeech: A Large Scale Mandarin Speech Corpus

Tingwei Guo, Wen Cheng+9 more|arXiv (Cornell University)|Oct 19, 2020

Speech Recognition and Synthesis52 references4 citations

TL;DR

DiDiSpeech is a large-scale, open-source Mandarin speech corpus comprising ~800 hours of high-quality, 48kHz audio from 6,000 native speakers, designed for diverse speech processing tasks. It enables state-of-the-art performance in automatic speech recognition (CER: 2.50%), voice conversion, multi-speaker TTS, and paralinguistic tasks like speaker classification (98.20% top-1 accuracy) and age estimation (MAE: 3.76), demonstrating its utility for both research and industrial applications.

ABSTRACT

This paper introduces a new open-sourced Mandarin speech corpus, called DiDiSpeech. It consists of about 800 hours of speech data at 48kHz sampling rate from 6000 speakers and the corresponding texts. All speech data in the corpus is recorded in quiet environment and is suitable for various speech processing tasks, such as voice conversion, multi-speaker text-to-speech and automatic speech recognition. We conduct experiments with multiple speech tasks and evaluate the performance, showing that it is promising to use the corpus for both academic research and practical application. The corpus is available at https://outreach.didichuxing.com/research/opendata/.

Motivation & Objective

• To address the lack of large-scale, high-quality, multi-purpose Mandarin speech corpora suitable for diverse speech processing tasks.
• To bridge the gap between academic research and industrial applications in Mandarin speech technology by providing a scalable, open-access dataset.
• To support advanced research in voice conversion, multi-speaker text-to-speech, automatic speech recognition, and paralinguistic information extraction (e.g., speaker, gender, and age classification).
• To ensure data quality through controlled recording conditions (quiet environments) and balanced demographic distribution across age, gender, and geography.
• To release a corpus that supports both parallel and non-parallel utterances for flexible model training and evaluation.

Proposed method

• The corpus is divided into two subsets: DiDiSpeech-1 (572 hours, 4,500 speakers) for voice conversion and multi-speaker TTS, and DiDiSpeech-2 (227 hours, 1,500 speakers) for ASR and paralinguistic tasks.
• All audio was recorded in quiet environments at 48kHz sampling rate, with corresponding transcriptions provided for all utterances.
• For ASR, models were trained using Transformer-ASR with beam search, language modeling, and CTC-joint decoding, achieving a CER of 2.50% on DiDiSpeech-2.
• For speaker and gender classification, a thin ResNet-34 architecture with 40- or 80-dimensional filter-bank features was used, followed by average pooling and a softmax layer for 6,000-class classification.
• For age estimation, the same model architecture was used with Mean Absolute Error (MAE) and Pearson correlation as evaluation metrics, trained with a smaller initial learning rate to ensure convergence.
• Data was split into training, development, and test sets by randomly selecting four utterances per speaker for dev and test sets, with no data augmentation or voice activity detection applied.

Figure 1: Speaker statistics.

Experimental results

Research questions

• RQ1 Can DiDiSpeech support high-performance automatic speech recognition in Mandarin, and how does it compare to existing benchmarks?
• RQ2 To what extent can DiDiSpeech enable effective voice conversion and multi-speaker text-to-speech synthesis in low-resource settings?
• RQ3 How well does DiDiSpeech support paralinguistic tasks such as speaker identification, gender classification, and age estimation?
• RQ4 Does the demographic diversity and data quality of DiDiSpeech lead to robust model generalization across age, gender, and regional accents?
• RQ5 Can DiDiSpeech serve as a viable alternative to proprietary industrial datasets in advancing Mandarin speech technology?

Key findings

• The DiDiSpeech-2 subset achieved a word-level CER of 2.50% on the test set using a Transformer-ASR model with CTC-joint decoding, demonstrating strong ASR performance.
• Speaker classification on DiDiSpeech achieved 98.20% top-1 accuracy and 99.76% top-5 accuracy on the test set, indicating high model reliability.
• Gender classification achieved 97.39% accuracy on both dev and test sets, showing strong performance across adult males, adult females, and children.
• Age estimation achieved a MAE of 3.76 and a Pearson correlation coefficient of 0.79 on the test set, indicating reasonable predictive accuracy for speaker age.
• The corpus supports diverse speech tasks including voice conversion, multi-speaker TTS, and speaker diarization, with consistent performance across all evaluated tasks.
• The inclusion of both parallel and non-parallel utterances in DiDiSpeech-1 enables flexible training for voice conversion and zero-shot transfer learning scenarios.

Figure 2: Text annotation example.