[Paper Review] LSTM-TDNN with convolutional front-end for Dialect Identification in the 2019 Multi-Genre Broadcast Challenge

Xiaoxiao Miao, Ian McLoughlin|arXiv (Cornell University)|Dec 19, 2019

Speech Recognition and Synthesis12 references4 citations

TL;DR

This paper proposes a CLSTM architecture combining a convolutional neural network front-end with a long short-term memory back-end for fine-grained Arabic dialect identification in the MGB-5 challenge. It achieves second place by leveraging time-scale modification (TSM) for data augmentation on low-resource dialects and traditional augmentation (TTDA), outperforming standard DNN x-vector systems through improved modeling of temporal dependencies and data balance.

ABSTRACT

This paper presents a novel Dialect Identification (DID) system developed for the Fifth Edition of the Multi-Genre Broadcast challenge, the task of Fine-grained Arabic Dialect Identification (MGB-5 ADI Challenge). The system improves upon traditional DNN x-vector performance by employing a Convolutional and Long Short Term Memory-Recurrent (CLSTM) architecture to combine the benefits of a convolutional neural network front-end for feature extraction and a back-end recurrent neural to capture longer temporal dependencies. Furthermore we investigate intensive augmentation of one low resource dialect in the highly unbalanced training set using time-scale modification (TSM). This converts an utterance to several time-stretched or time-compressed versions, subsequently used to train the CLSTM system without using any other corpus. In this paper, we also investigate speech augmentation using MUSAN and the RIR datasets to increase the quantity and diversity of the existing training data in the normal way. Results show firstly that the CLSTM architecture outperforms a traditional DNN x-vector implementation. Secondly, adopting TSM-based speed perturbation yields a small performance improvement for the unbalanced data, finally that traditional data augmentation techniques yield further benefit, in line with evidence from related speaker and language recognition tasks. Our system achieved 2nd place ranking out of 15 entries in the MGB-5 ADI challenge, presented at ASRU 2019.

Motivation & Objective

• Address the challenge of fine-grained Arabic dialect identification (DID) in highly unbalanced, low-resource training data.
• Improve upon traditional DNN x-vector systems by integrating a CNN-LSTM hybrid architecture to better model temporal and spectral features.
• Mitigate data imbalance in low-resource dialects like Jordanian (JOR) without using external corpora, by applying time-scale modification (TSM).
• Evaluate the effectiveness of both TSM and traditional training data augmentation (TTDA) in improving generalization and reducing error rates.
• Achieve state-of-the-art performance in the MGB-5 ADI challenge under strict data usage constraints.

Proposed method

• Employ a CLSTM architecture: a CNN front-end extracts local, context-aware features from speech frames, followed by an LSTM layer to model long-term temporal dependencies.
• Use time-scale modification (TSM) via the RTISI algorithm to generate time-stretched or compressed versions of low-resource dialect utterances, increasing training data diversity.
• Apply TSM exclusively to the most underrepresented dialect (JOR) to balance the training set without using external data, complying with challenge rules.
• Combine TSM with traditional data augmentation (TTDA) using MUSAN noise and RIR reverberation datasets to further enhance data variability and model robustness.
• Train the CLSTM model using end-to-end backpropagation with a cross-entropy loss function, optimizing for segment-level classification.
• Use statistical pooling (mean and standard deviation) to produce fixed-dimension embeddings from frame-level outputs before final classification.

Experimental results

Research questions

• RQ1 Can a CNN-LSTM hybrid architecture (CLSTM) outperform standard DNN x-vector systems in fine-grained Arabic dialect identification?
• RQ2 To what extent does time-scale modification (TSM) improve performance on low-resource dialects when no external data is allowed?
• RQ3 How effective is traditional data augmentation (TTDA) with noise and reverberation in improving DID performance across all dialects?
• RQ4 Does the combination of TSM and TTDA yield synergistic improvements in model generalization and error reduction?
• RQ5 How do confusion patterns between similar dialects (e.g., Levantine Arabic) reflect the model’s ability to capture subtle acoustic and phonological differences?

Key findings

• The CLSTM architecture achieved 93.06% test set accuracy and 2.09% EER, outperforming the baseline DNN x-vector system.
• TSM-based augmentation on the low-resource Jordanian (JOR) dialect improved performance, achieving 80.17% accuracy for JOR despite its small size.
• Combining TSM and TTDA yielded the best overall test set performance: 93.06% accuracy and 2.09% EER, slightly outperforming individual methods.
• The Lebanese (LEB) dialect achieved the highest accuracy at 96.97%, indicating strong model discrimination for well-represented dialects.
• Confusion was highest among Levantine dialects (e.g., JOR, PAL, SYR, LEB), suggesting shared phonological and lexical features increase misclassification risk.
• Despite high similarity among Levantine dialects, the model showed reduced confusion in some pairs, possibly due to subtle acoustic and phonetic differences captured by the CLSTM architecture.