[Paper Review] Triplet Based Embedding Distance and Similarity Learning for Text-independent Speaker Verification

Zongze Ren, Zhiyong Chen+1 more|arXiv (Cornell University)|Aug 6, 2019

Speech Recognition and Synthesis17 references4 citations

TL;DR

This paper proposes a triplet-based training framework for text-independent speaker verification that jointly optimizes triplet distance loss and an embedding similarity measurement network with the standard softmax loss. By introducing a dual-branch DNN to classify speaker similarity and refining embedding distances, the method achieves a 9% relative reduction in equal error rate (EER) and detected cost function (DCF) on the NIST SRE16 test set, outperforming the baseline x-vector system.

ABSTRACT

Speaker embeddings become growing popular in the text-independent speaker verification task. In this paper, we propose two improvements during the training stage. The improvements are both based on triplet cause the training stage and the evaluation stage of the baseline x-vector system focus on different aims. Firstly, we introduce triplet loss for optimizing the Euclidean distances between embeddings while minimizing the multi-class cross entropy loss. Secondly, we design an embedding similarity measurement network for controlling the similarity between the two selected embeddings. We further jointly train the two new methods with the original network and achieve state-of-the-art. The multi-task training synergies are shown with a 9% reduction equal error rate (EER) and detected cost function (DCF) on the 2016 NIST Speaker Recognition Evaluation (SRE) Test Set.

Motivation & Objective

• To improve text-independent speaker verification performance by addressing the discrepancy between training and evaluation objectives in x-vector systems.
• To reduce embedding variance within speakers and increase separation between speakers using triplet-based optimization.
• To enhance scoring robustness by introducing a learnable similarity measurement network for utterance embeddings.
• To jointly train multiple loss functions (softmax, triplet distance, and similarity) for synergistic performance gains.

Proposed method

• Introduces a triplet loss that minimizes the Euclidean distance between anchor and positive embeddings while maximizing the distance to negative embeddings, directly optimizing for verification performance.
• Designs a secondary DNN, the embedding similarity measurement network, which takes concatenated anchor and positive/negative embeddings as input and predicts their speaker similarity.
• Jointly trains the x-vector backbone with three loss functions: multi-class cross-entropy (softmax), triplet distance loss, and similarity classification loss, using learnable weighting coefficients β and γ.
• Applies data augmentation using MUSAN and RIRs to increase training set diversity and robustness.
• Uses t-SNE visualization and DET curves to analyze embedding clustering and score distributions across systems.
• Employs Kaldi’s PLDA backend for scoring and evaluates on the NIST SRE16 test set using EER and DCF16 metrics.

Experimental results

Research questions

• RQ1 Can triplet-based distance optimization reduce within-speaker variance and improve EER in text-independent speaker verification?
• RQ2 Can a learned similarity measurement network improve DCF by better modeling inter-embedding similarity?
• RQ3 Does joint training of multiple losses (softmax, triplet distance, and similarity) yield synergistic performance gains over single-loss or dual-loss training?
• RQ4 How do different embedding layers (A vs. B) and normalization techniques affect the performance of the proposed methods?

Key findings

• The proposed system with joint training of softmax, triplet distance, and similarity loss (System 5) achieved a 9% relative reduction in equal error rate (EER) and detected cost function (DCF) on the SRE16 test set.
• The embedding similarity measurement network alone (System 3) significantly reduced DCF by 10.5% while slightly increasing EER, indicating improved backend scoring performance.
• Triplet distance training alone (System 4) achieved the largest EER reduction (12.5% relative), demonstrating improved within-speaker compactness.
• Joint training with both triplet distance and similarity networks (System 5) outperformed individual components, showing complementary benefits and enabling trade-offs via hyperparameter tuning (β and γ).
• The similarity network’s non-target score distribution was more negative than baseline systems, confirming improved discriminative power.
• L2 normalization did not improve performance (System 7), and using embedding B instead of embedding A yielded worse results (System 8), indicating embedding A is more suitable for the proposed modules.