[Paper Review] A Compact End-to-End Model with Local and Global Context for Spoken Language Identification

Fei Jia, Nithin Rao Koluguri+2 more|arXiv (Cornell University)|Oct 27, 2022

Speech Recognition and Synthesis4 citations

TL;DR

TitaNet-LID is a compact end-to-end neural network for spoken language identification that combines 1D depth-wise separable convolutions and Squeeze-and-Excitation modules to capture both local and global context. Despite being 10× smaller than state-of-the-art SSL models, it achieves state-of-the-art accuracy of 88.2% on the FLEURS benchmark and performs robustly even on short utterances (<5s) through simple fine-tuning.

ABSTRACT

We introduce TitaNet-LID, a compact end-to-end neural network for Spoken Language Identification (LID) that is based on the ContextNet architecture. TitaNet-LID employs 1D depth-wise separable convolutions and Squeeze-and-Excitation layers to effectively capture local and global context within an utterance. Despite its small size, TitaNet-LID achieves performance similar to state-of-the-art models on the VoxLingua107 dataset while being 10 times smaller. Furthermore, it can be easily adapted to new acoustic conditions and unseen languages through simple fine-tuning, achieving a state-of-the-art accuracy of 88.2% on the FLEURS benchmark. Our model is scalable and can achieve a better trade-off between accuracy and speed. TitaNet-LID performs well even on short utterances less than 5s in length, indicating its robustness to input length.

Motivation & Objective

• To develop a compact, end-to-end neural network for spoken language identification (LID) that maintains high accuracy while minimizing model size.
• To enable effective adaptation to unseen languages and new acoustic conditions through simple fine-tuning.
• To achieve strong performance on short speech segments (<5s), which are common in real-world applications.
• To provide a scalable architecture that balances accuracy and inference speed for deployment in compute-constrained environments.
• To demonstrate that a lightweight model without self-supervised pretraining can match or exceed the performance of much larger SSL-based models on LID benchmarks.

Proposed method

• The model uses a ContextNet-inspired encoder with 1D depth-wise separable convolutions to extract local features efficiently.
• Squeeze-and-Excitation (SE) layers are integrated to model global context by recalibrating channel-wise feature responses.
• The encoder is followed by a statistic pooling layer and two fully connected layers to produce a fixed-length utterance-level representation.
• The architecture is scalable: model depth is adjusted via the number of repeated blocks (R), and width via the number of filters (C).
• The model is trained end-to-end from scratch on VoxLingua107 and fine-tuned on FLEURS for zero-shot generalization to unseen languages.
• The implementation is released via NVIDIA NeMo, enabling easy deployment and adaptation.

Figure 1: TitaNet-LID- $B$ x $R$ x $C$ Encoder is based on ContextNet architecture [ 20 ] , where $B$ is the number of blocks, $R$ is the number of repeated ``basic” blocks and $C$ is the number of filters in the convolution layers of each block.

Experimental results

Research questions

• RQ1 Can a compact, end-to-end model with local and global context mechanisms achieve state-of-the-art performance on spoken language identification without self-supervised pretraining?
• RQ2 How well does the model generalize to unseen languages and new acoustic conditions when fine-tuned on limited data?
• RQ3 What is the impact of input segment length on model accuracy, especially for short utterances (<5s)?
• RQ4 Can the model be effectively scaled in size to balance accuracy and inference speed for different deployment scenarios?
• RQ5 How does the performance of a lightweight model compare to large SSL-based models in terms of accuracy and parameter efficiency?

Key findings

• TitaNet-LID-3x5x1024 (29M parameters) achieves 88.2% macro accuracy on the FLEURS benchmark, outperforming the previous SOTA model by 16.8%.
• The model achieves 95.0% macro accuracy on VoxLingua107 with 28.9M parameters, outperforming the 21M-parameter ECAPA-TDNN-LID baseline.
• On short utterances (<5s), TitaNet-LID maintains strong performance, demonstrating robustness to input length variation.
• Fine-tuning on the union of FLEURS and VoxLingua107 enables the model to predict 127 languages with 93.8% macro accuracy.
• The model is 10× smaller than the XLS-R (0.3B) model while achieving comparable performance on VoxLingua107, and 20× smaller than w2v-bert-51 (0.6B) while outperforming it on FLEURS.
• Scaling the model by increasing filter count (C) and block repetition (R) improves accuracy, with TitaNet-LID-3x5x2048 achieving 95.9% macro accuracy.

Figure 2: Number of samples for each language in VoxLingua107 evaluation set. Dark blue represents accurately identified samples of TitaNet-LID-3x5x1024.