[Paper Review] Deep Neural Network Embedding Learning with High-Order Statistics for Text-Independent Speaker Verification.

Lanhua You, Wu Guo+2 more|arXiv (Cornell University)|Mar 28, 2019

Speech Recognition and Synthesis12 references4 citations

TL;DR

This paper proposes a multi-task learning framework for x-vector speaker embeddings that jointly optimizes speaker classification and reconstruction of high-order statistics from raw utterances. By integrating supervised and unsupervised signals, the method enhances embedding discriminability and robustness with minimal computational overhead, achieving superior performance on NIST SRE16 and VOiCES datasets compared to standard x-vector systems.

ABSTRACT

The x-vector based deep neural network (DNN) embedding systems have demonstrated effectiveness for text-independent speaker verification. This paper presents a multi-task learning architecture for training the speaker embedding DNN, with the primary task of classifying the target speakers and the auxiliary task of reconstructing the higher-order statistics of the original input utterance. The proposed training strategy aggregates both the supervised and unsupervised learning into one framework to make the speaker embeddings more discriminative and robust. Experiments are carried out in the NIST SRE16 evaluation dataset and the VOiCES dataset. The results demonstrate that our proposed method outperform the original x-vector approach with very low additional complexity added.

Motivation & Objective

• To improve text-independent speaker verification performance by leveraging both supervised and unsupervised learning signals.
• To address the limitation of standard x-vector systems that rely solely on speaker classification for embedding learning.
• To enhance embedding robustness and discriminativeness through auxiliary reconstruction of higher-order statistics of input utterances.
• To develop a training framework that integrates supervised and unsupervised objectives without significantly increasing model complexity.

Proposed method

• Introduces a multi-task learning architecture where the primary task is speaker classification and the auxiliary task is reconstructing higher-order statistics of the input utterance.
• Uses the higher-order statistics (e.g., third- and fourth-order moments) of the raw acoustic features as a supervisory signal during training.
• Trains the DNN to jointly optimize speaker identity classification and reconstruction of these statistical features using a shared encoder backbone.
• Applies a reconstruction loss on the high-order statistics to encourage the network to preserve discriminative and robust representations.
• Employs a shared bottleneck layer (x-vector) for speaker embedding extraction, shared across both tasks.
• The training objective combines cross-entropy loss for speaker classification and mean squared error loss for statistical reconstruction.

Experimental results

Research questions

• RQ1 Can reconstructing higher-order statistics of utterances improve the discriminativeness of speaker embeddings in text-independent verification?
• RQ2 How does joint learning of speaker classification and statistical reconstruction affect robustness to channel and environmental variations?
• RQ3 What is the impact of adding an unsupervised auxiliary task on the performance of x-vector systems with minimal computational cost?
• RQ4 Does the proposed multi-task framework generalize across diverse evaluation sets such as NIST SRE16 and VOiCES?

Key findings

• The proposed method achieves better performance than the standard x-vector system on the NIST SRE16 evaluation dataset.
• The method also outperforms the baseline on the VOiCES dataset, demonstrating robustness across diverse recording conditions.
• The improvement is achieved with very low additional computational complexity, making it practical for real-world deployment.
• The integration of high-order statistics as an auxiliary task enhances the discriminative power of learned speaker embeddings.
• The results confirm that unsupervised signal from higher-order statistics contributes meaningfully to embedding quality without requiring additional labels.