[Paper Review] Pairwise Discriminative Neural PLDA for Speaker Verification

Shreyas Ramoji, Prashant Krishnan+2 more|arXiv (Cornell University)|Jan 20, 2020

Speech Recognition and Synthesis21 references4 citations

TL;DR

This paper proposes a pairwise discriminative neural PLDA model that jointly learns pre-processing (LDA, normalization) and scoring for speaker verification using x-vectors. By differentiating through these layers and optimizing a soft approximation of the NIST SRE detection cost, the model achieves 30% relative improvement in CMN2 and 30% in VAST conditions over the PLDA baseline, demonstrating superior generalization and reduced overfitting.

ABSTRACT

The state-of-art approach to speaker verification involves the extraction of discriminative embeddings like x-vectors followed by a generative model back-end using a probabilistic linear discriminant analysis (PLDA). In this paper, we propose a Pairwise neural discriminative model for the task of speaker verification which operates on a pair of speaker embeddings such as x-vectors/i-vectors and outputs a score that can be considered as a scaled log-likelihood ratio. We construct a differentiable cost function which approximates speaker verification loss, namely the minimum detection cost. The pre-processing steps of linear discriminant analysis (LDA), unit length normalization and within class covariance normalization are all modeled as layers of a neural model and the speaker verification cost functions can be back-propagated through these layers during training. We also explore regularization techniques to prevent overfitting, which is a major concern in using discriminative back-end models for verification tasks. The experiments are performed on the NIST SRE 2018 development and evaluation datasets. We observe average relative improvements of 8% in CMN2 condition and 30% in VAST condition over the PLDA baseline system.

Motivation & Objective

• To address overfitting in discriminative back-ends for speaker verification by using a task-specific loss function.
• To unify pre-processing steps (LDA, normalization) and scoring into an end-to-end trainable neural architecture.
• To improve generalization on unseen data by replacing binary cross-entropy with a soft approximation of the NIST SRE detection cost.
• To explore parameter initialization from generative PLDA to enhance performance.
• To demonstrate that a parameter-efficient neural PLDA can outperform standard PLDA and DPLDA on SRE 2018 datasets.

Proposed method

• The model takes a pair of x-vectors (enrollment and test) as input and outputs a score interpreted as a scaled log-likelihood ratio.
• Pre-processing steps—LDA, unit length normalization, and WCCN—are modeled as differentiable neural layers, enabling backpropagation through the entire pipeline.
• A differentiable soft approximation of the NIST SRE minimum detection cost (C_min) is used as the primary loss function to prioritize false alarm reduction.
• The model is trained with batch sizes of 4096–8192 and a learning rate schedule that decays when validation loss plateaus.
• The network is initialized either randomly or with parameters from a Kaldi PLDA baseline to improve convergence and performance.
• Regularization is applied via architectural constraints and the choice of loss function to prevent overfitting to training speakers.

Experimental results

Research questions

• RQ1 Can a neural back-end that jointly learns pre-processing and scoring improve generalization in speaker verification compared to standard PLDA?
• RQ2 Does optimizing for the NIST SRE detection cost instead of binary cross-entropy reduce overfitting and improve performance on unseen data?
• RQ3 How does parameter initialization from a generative PLDA model affect the performance of a discriminative neural PLDA?
• RQ4 What is the impact of architectural design and regularization on model generalization in discriminative speaker verification back-ends?
• RQ5 Can a differentiable neural model that embeds LDA, normalization, and scoring outperform traditional PLDA and DPLDA on SRE 2018 benchmarks?

Key findings

• The proposed neural PLDA model achieves a 10% relative improvement in C_min on the SRE 2018 development set under CMN2 conditions compared to the PLDA baseline.
• On the VAST condition, the model achieves a 38% relative improvement in C_min on the development set, demonstrating strong generalization.
• On the SRE 2018 evaluation set, the model yields 7% relative improvement in CMN2 and 23% in VAST conditions over the PLDA baseline.
• Using the soft detection cost loss function leads to significantly better generalization than binary cross-entropy, especially on out-of-domain conditions.
• Initializing the neural PLDA with Kaldi PLDA parameters further improves performance, indicating that prior generative knowledge enhances discriminative learning.
• The model with fewer parameters generalizes better than DPLDA, suggesting that reduced degrees of freedom help mitigate overfitting.