[Paper Review] AP16-OL7: A Multilingual Database for Oriental Languages and A Language Recognition Baseline

Dong Wang, Lantian Li+2 more|arXiv (Cornell University)|Sep 27, 2016

Natural Language Processing Techniques9 references3 citations

TL;DR

This paper presents AP16-OL7, a multilingual speech database for seven Oriental languages (Mandarin, Cantonese, Indonesian, Japanese, Russian, Korean, Vietnamese), collected via mobile phones with balanced male and female speakers. A baseline language recognition system using i-vector and LDA features with SVM scoring achieved a C_avg of 3.37% and IDR of 91.99% with RBF kernel, demonstrating the database's suitability for multilingual research and language recognition tasks in low-resource settings.

ABSTRACT

We present the AP16-OL7 database which was released as the training and test data for the oriental language recognition (OLR) challenge on APSIPA 2016. Based on the database, a baseline system was constructed on the basis of the i-vector model. We report the baseline results evaluated in various metrics defined by the AP16-OLR evaluation plan and demonstrate that AP16-OL7 is a reasonable data resource for multilingual research.

Motivation & Objective

• To address the scarcity of multilingual speech resources for low-resource Oriental languages, particularly in East, Southeast, and Northeast Asia.
• To support the APSIPA 2016 Oriental Language Recognition (OLR) challenge by providing a standardized training and test database.
• To establish a robust baseline system for language recognition using i-vector and LDA-based features on multilingual speech data.
• To evaluate system performance using multiple metrics (C_avg, EER, minDCF, DET, IDR) to ensure comprehensive assessment of recognition capability.
• To demonstrate that AP16-OL7 is a viable and representative data resource for multilingual speech processing research in low-resource language settings.

Proposed method

• The AP16-OL7 database comprises 7 language-specific datasets, each with 24 speakers (12 male, 12 female), recording ~10 hours of speech per language via mobile phones at 16 kHz, 16-bit resolution.
• Each dataset is split into a training set (18 speakers) and a test set (6 speakers), with utterances collected in a reading style to ensure consistency.
• The baseline system employs i-vector extraction from speech features, followed by LDA dimensionality reduction to enhance language discriminability.
• Scoring is performed using both cosine distance and SVM-based classification with three kernel types: linear, polynomial (degree 3), and RBF.
• T-SNE visualization is used to analyze the clustering behavior of i-vectors before and after LDA transformation, assessing speaker and language separation.
• Performance is evaluated using multiple metrics: C_avg (cost-averaged error rate), EER (equal error rate), minDCF (minimum detection cost function), DET curves, and IDR (identification rate).

Experimental results

Research questions

• RQ1 Can AP16-OL7 serve as a reliable and representative multilingual speech database for low-resource Oriental languages in language recognition tasks?
• RQ2 How effective is the i-vector + LDA + SVM baseline system in distinguishing between seven diverse Oriental languages with varying linguistic and phonetic characteristics?
• RQ3 To what extent does LDA transformation improve language discriminability in i-vector space, as visualized via T-SNE?
• RQ4 Which scoring method—cosine distance or SVM with different kernels—yields the best performance across multiple evaluation metrics?
• RQ5 How do the evaluation metrics (C_avg, EER, minDCF, DET, IDR) collectively reflect the robustness and reliability of the baseline system?

Key findings

• The AP16-OL7 database is the first multilingual speech database specifically designed for Oriental languages, with balanced, high-quality recordings across seven languages.
• The i-vector + LDA + RBF-SVM baseline achieved the lowest C_avg (3.40%) and highest IDR (92.04%) among all configurations, indicating strong language recognition performance.
• LDA transformation significantly improved language discriminability, as shown by T-SNE visualizations, where language clusters became more distinct after LDA projection.
• SVM-based scoring consistently outperformed cosine distance scoring across all metrics, with the RBF kernel yielding the best results.
• The minimum detection cost function (minDCF) was reduced to 0.0333 with the L-vector-SVM (RBF) configuration, indicating high-quality scoring performance.
• The baseline system achieved an EER of 3.36% with the L-vector-SVM (RBF) setup, demonstrating strong verification capability on the multilingual test set.