[Paper Review] Deep Discriminant Analysis for i-vector Based Robust Speaker Recognition

Shuai Wang, Zili Huang+2 more|arXiv (Cornell University)|May 3, 2018

Speech Recognition and Synthesis25 references3 citations

TL;DR

This paper proposes Deep Discriminant Analysis (DDA), a non-linear neural network-based compensation method for i-vector speaker recognition that jointly optimizes softmax and center loss to enhance discriminative power. Unlike linear LDA, DDA learns a non-linear projection that reduces intra-speaker variation and increases inter-speaker separation, achieving a 4.69% EER with Euclidean scoring—surpassing both LDA and PLDA on the SRE corpus dataset.

ABSTRACT

Linear Discriminant Analysis (LDA) has been used as a standard post-processing procedure in many state-of-the-art speaker recognition tasks. Through maximizing the inter-speaker difference and minimizing the intra-speaker variation, LDA projects i-vectors to a lower-dimensional and more discriminative sub-space. In this paper, we propose a neural network based compensation scheme(termed as deep discriminant analysis, DDA) for i-vector based speaker recognition, which shares the spirit with LDA. Optimized against softmax loss and center loss at the same time, the proposed method learns a more compact and discriminative embedding space. Compared with the Gaussian distribution assumption of data and the learnt linear projection in LDA, the proposed method doesn't pose any assumptions on data and can learn a non-linear projection function. Experiments are carried out on a short-duration text-independent dataset based on the SRE Corpus, noticeable performance improvement can be observed against the normal LDA or PLDA methods.

Motivation & Objective

• Address the limitations of linear LDA in i-vector speaker recognition, particularly its assumptions about Gaussian data distribution and inability to model non-linear separability.
• Develop a non-linear alternative to LDA that better captures complex speaker variability and channel degradation in real-world conditions.
• Improve speaker recognition performance on short-duration, text-independent utterances by learning a more compact and discriminative embedding space.
• Demonstrate that a neural network-based approach can surpass traditional PLDA and LDA systems when combined with simple scoring methods like cosine or Euclidean distance.

Proposed method

• Propose a deep neural network that maps high-dimensional i-vectors (600D) to a lower-dimensional discriminative space (e.g., 300D or 400D) using a multi-layer architecture with PReLU activations and batch normalization.
• Train the network using a joint loss function combining softmax loss (to maximize inter-class separation) and center loss (to minimize intra-class variation by pulling embeddings toward class centers).
• Optimize the network with separate learning rates: 0.01 for the main network and 0.1 for the center parameters, with centers updated per mini-batch to improve training stability.
• Use a weighted combination of losses with λ = 0.01 to balance the two objectives, ensuring the network learns meaningful discriminative features without collapsing into a non-distinguishable embedding space.
• Apply t-SNE visualization to qualitatively assess the compactness of intra-speaker embeddings and the separation between speakers after DDA transformation.
• Evaluate the method on the SRE corpus using both cosine and Euclidean scoring, comparing against baseline i-vector, LDA, and PLDA systems.

Experimental results

Research questions

• RQ1 Can a deep neural network effectively replace linear LDA in i-vector space by learning a non-linear, more discriminative projection without assuming Gaussian data distribution?
• RQ2 How does joint optimization of softmax and center loss improve speaker recognition performance compared to standard LDA or PLDA?
• RQ3 What is the optimal dimensionality of the DDA embedding space for achieving minimal EER on short-duration, text-independent speaker recognition tasks?
• RQ4 How sensitive is the DDA model to the hyperparameter λ balancing softmax and center loss, and what is the optimal setting for convergence and performance?

Key findings

• DDA achieves an EER of 4.69% using Euclidean scoring, outperforming the baseline PLDA system (EER 4.96%) and LDA (EER 5.22%) on the SRE corpus.
• The best performance for DDA is achieved at a 400-dimensional embedding space, where EER drops to 4.51% with Euclidean scoring, while LDA performs best at 200D.
• When λ is set to 0.01, training converges stably and yields optimal performance; values like 0.1 make the network untrainable due to excessive center loss dominance.
• t-SNE visualization confirms that DDA significantly reduces intra-speaker variation, making embeddings from the same speaker more compact and better separated from other speakers.
• DDA outperforms LDA across all embedding dimensions tested, with the largest relative improvement observed in Euclidean scoring (e.g., 4.69% EER vs. 5.22% for LDA at 300D).
• The proposed method is incompatible with PLDA, suggesting that DDA’s non-linear compensation may already subsume the role of PLDA in the embedding space.