[Paper Review] The DKU-MSXF Speaker Verification System for the VoxCeleb Speaker Recognition Challenge 2023

Ze Li, Yuke Lin+4 more|arXiv (Cornell University)|Aug 17, 2023

Speech Recognition and Synthesis4 citations

TL;DR

This paper presents the DKU-MSXF system for the VoxCeleb Speaker Recognition Challenge 2023, introducing a cross-age QMF training strategy for improved performance in fully supervised tracks and a novel pseudo-labeling method with triple thresholds and sub-center purification for semi-supervised domain adaptation in Track 3. The system achieved an mDCF of 0.1243 (Track 1), 0.1165 (Track 2), and EER of 4.952% (Track 3).

ABSTRACT

This paper is the system description of the DKU-MSXF System for the track1, track2 and track3 of the VoxCeleb Speaker Recognition Challenge 2023 (VoxSRC-23). For Track 1, we utilize a network structure based on ResNet for training. By constructing a cross-age QMF training set, we achieve a substantial improvement in system performance. For Track 2, we inherite the pre-trained model from Track 1 and conducte mixed training by incorporating the VoxBlink-clean dataset. In comparison to Track 1, the models incorporating VoxBlink-clean data exhibit a performance improvement by more than 10% relatively. For Track3, the semi-supervised domain adaptation task, a novel pseudo-labeling method based on triple thresholds and sub-center purification is adopted to make domain adaptation. The final submission achieves mDCF of 0.1243 in task1, mDCF of 0.1165 in Track 2 and EER of 4.952% in Track 3.

Motivation & Objective

• To improve speaker verification performance in fully supervised settings using cross-age data augmentation.
• To enhance model generalization by incorporating the large-scale VoxBlink-clean dataset via mixed fine-tuning.
• To address domain shift in semi-supervised speaker recognition using a novel pseudo-labeling strategy with threshold-based clustering and sub-center purification.
• To achieve state-of-the-art performance on VoxCeleb Speaker Recognition Challenge 2023 tracks 1, 2, and 3.

Proposed method

• A ResNet100-based backbone with SimAM and fwSE modules is used for speaker embedding extraction.
• Cross-age QMF training is applied to improve generalization across age groups in Track 1.
• For Track 2, mixed fine-tuning (Mix-FT) is applied using both VoxCeleb2 and VoxBlink-clean data.
• A three-stage pseudo-labeling pipeline is proposed: KNN graph construction with T1, class purity estimation using T2, and class merging with T3.
• Sub-center purification is applied to remove low-purity classes by analyzing selection probabilities from a sub-center ArcFace classifier.
• Score calibration and normalization are performed using QMF and a cohort of 20,000 unlabeled target-domain utterances.

Experimental results

Research questions

• RQ1 How does cross-age QMF training improve speaker verification performance in fully supervised settings?
• RQ2 To what extent does incorporating the VoxBlink-clean dataset via mixed fine-tuning enhance model robustness and accuracy?
• RQ3 Can a threshold-based pseudo-labeling strategy with sub-center purification effectively handle domain shift in semi-supervised speaker recognition?
• RQ4 How does the proposed method compare to prior approaches on the VoxSRC-2023 challenge, particularly in Track 3?

Key findings

• The DKU-MSXF system achieved an mDCF of 0.1243 on the Track 1 evaluation set, representing a significant improvement over baseline systems.
• In Track 2, the system achieved an mDCF of 0.1165, demonstrating over 10% relative performance gain compared to the Track 1 baseline.
• For Track 3, the system achieved an EER of 4.952% on the official evaluation set, outperforming the previous year's first-place result (EER 7.03%).
• The fusion of multiple models in Track 1 and Track 2 reduced the EER to 2.079% and 2.029%, respectively, on the validation set.
• The sub-center purification step effectively reduced noisy class assignments, improving the reliability of pseudo-labeled data in semi-supervised training.
• The use of Mix-FT with VoxBlink-clean data led to a relative improvement of over 10% in Track 2 compared to training from scratch.