[Paper Review] UNISOUND System for VoxCeleb Speaker Recognition Challenge 2023

Yufeng Zheng, Yajun Zhang+4 more|arXiv (Cornell University)|Aug 24, 2023

Speech Recognition and Synthesis4 citations

TL;DR

This paper presents the UNISOUND system, which achieved first place in Track 1 and second in Track 2 of the VoxSRC 2023 speaker recognition challenge. It introduces a consistency-aware score calibration method using a Consistency Measure Factor (CMF) that leverages embedding stability across audio crops, combined with large-scale ResNet and RepVGG backbones, data augmentation, and score fusion, achieving a minDCF of 0.0855 and EER of 1.5880%.

ABSTRACT

This report describes the UNISOUND submission for Track1 and Track2 of VoxCeleb Speaker Recognition Challenge 2023 (VoxSRC 2023). We submit the same system on Track 1 and Track 2, which is trained with only VoxCeleb2-dev. Large-scale ResNet and RepVGG architectures are developed for the challenge. We propose a consistency-aware score calibration method, which leverages the stability of audio voiceprints in similarity score by a Consistency Measure Factor (CMF). CMF brings a huge performance boost in this challenge. Our final system is a fusion of six models and achieves the first place in Track 1 and second place in Track 2 of VoxSRC 2023. The minDCF of our submission is 0.0855 and the EER is 1.5880%.

Motivation & Objective

• To develop a high-performance speaker recognition system for the VoxSRC 2023 challenge using only VoxCeleb2-dev data.
• To improve system robustness and accuracy through advanced model architectures and data augmentation.
• To introduce a novel consistency-aware score calibration method that enhances similarity score reliability.
• To achieve state-of-the-art performance on both Track 1 (closed-set) and Track 2 (open-set) evaluation protocols.

Proposed method

• Trained large-scale ResNet (up to 518 layers) and RepVGG-B1 backbones using only VoxCeleb2-dev data, with 3-fold speed augmentation and RIR/MUSAN-based data augmentation.
• Extracted 80-dimensional log Mel filter bank features with mean normalization, without voice activity detection.
• Employed Multi-query Multi-head Attention (MQMHA) pooling with 4 queries and 16 heads, using only standard deviation for ResNets and both mean and std for RepVGG.
• Applied AM-Softmax and AAM-Softmax loss functions in two-stage training with margin scheduling and learning rate decay.
• Proposed Consistency Measure Factor (CMF) as a score calibration method: CMF = (1/N) * ||Σ(x_i / ||x_i||)||, measuring embedding consistency across temporal crops.
• Applied AS-Norm and QMF (Quality-aware Model Fusion) to further calibrate and fuse scores from multiple models.

Experimental results

Research questions

• RQ1 Can a single system trained only on VoxCeleb2-dev achieve top-tier performance in the VoxSRC 2023 challenge without external data?
• RQ2 How does the Consistency Measure Factor (CMF) improve speaker recognition performance by quantifying embedding stability across audio crops?
• RQ3 What is the impact of using only standard deviation in the pooling layer for ResNet-based models compared to mean and standard deviation?
• RQ4 How effective is the combination of CMF, AS-Norm, and QMF in boosting system performance on both closed-set and open-set evaluation sets?
• RQ5 Can large-scale deep networks like ResNet518 and RepVGG-B1 outperform smaller models in speaker verification with minimal data?

Key findings

• The final system, a fusion of six models (ResNet314, ResNet518, and RepVGG-B1), achieved a minDCF of 0.0855 and EER of 1.5880% on the VoxSRC2023 test set.
• The CMF-based calibration method reduced minDCF by 6.6% compared to baseline backend methods, significantly improving performance on the dev set.
• Using CMF with 2-second audio crops outperformed segment scoring with 2-second crops, reducing minDCF from 0.1190 to 0.1080.
• The CMF method improved performance on short-duration audio (2s) by 1.3% in EER and 1.1% in minDCF compared to segment scoring.
• The fusion of models with CMF, AS-Norm, and QMF achieved 1.760% EER and 0.0993 minDCF on the VoxSRC2023 test set, outperforming all baselines.
• The system achieved first place in Track 1 and second place in Track 2 of VoxSRC 2023, with relative improvements of 12.1% and 6.8% in minDCF on VoxCeleb1-E and VoxCeleb1-H compared to prior work.