[Paper Review] Clova Baseline System for the VoxCeleb Speaker Recognition Challenge 2020

Hee Soo Heo, Bong‐Jin Lee+2 more|arXiv (Cornell University)|Sep 29, 2020

Speech Recognition and Synthesis18 references96 citations

TL;DR

The paper presents Clova’s ResNet-based baseline for VoxSRC 2020, exploring multiple loss functions and pooling mechanisms to improve speaker recognition without ensembles or post-processing. It reports strong single-model results and releases training code and models as unofficial baselines.

ABSTRACT

This report describes our submission to the VoxCeleb Speaker Recognition Challenge (VoxSRC) at Interspeech 2020. We perform a careful analysis of speaker recognition models based on the popular ResNet architecture, and train a number of variants using a range of loss functions. Our results show significant improvements over most existing works without the use of model ensemble or post-processing. We release the training code and pre-trained models as unofficial baselines for this year's challenge.

Motivation & Objective

• Assess effective ResNet-based architectures for speaker recognition on VoxCeleb data under VoxSRC 2020 conditions (domain shift, short utterances).
• Investigate how different loss functions (AM-Softmax, AAM-Softmax, AP, AP+Softmax) and pooling (SAP, ASP) affect performance.
• Evaluate input representations and augmentation strategies to improve robustness in unconstrained speech.
• Provide an unofficial baseline by releasing training code and pretrained models for the community.

Proposed method

• Use fixed-length 2-second segments with 64-dim log-Mel filterbanks and MVN via instance normalization as input features.
• Compare two ResNet-34 variants: a speed-optimized model with 1.4M parameters and SAP, and a performance-optimized model with 8.0M parameters and ASP.
• Apply metric-learning and classification-based losses (AM-Softmax, AAM-Softmax, AP, AP+Softmax) to learn discriminative speaker embeddings.
• Aggregate frame-level features using attention-based pooling (SAP/ASP) to obtain utterance-level representations.
• Train on VoxCeleb2 development set and evaluate with cosine similarity across 10 crops per test segment, averaging 100 pairwise similarities.
• Employ data augmentation including MUSAN noise/music/other noises and simulated room impulse responses (RIRs).
• Implement in PyTorch on the NSML platform with distributed training across multiple GPUs; report results without model ensembling.

Experimental results

Research questions

• RQ1 Can ResNet-based backbones with attention-based pooling match or exceed prior VoxSRC baselines without model ensembling?
• RQ2 How do AM-Softmax, AAM-Softmax, AP, and AP+Softmax losses compare for speaker verification in unconstrained conditions?
• RQ3 What is the impact of using SAP vs ASP on utterance-level discrimination in VoxCeleb data?
• RQ4 Does embedding batch normalization (BN) on the embeddings improve performance for classification objectives?

Key findings

• A speed-optimized Q/SAP model and a performance-optimized H/ASP model were developed and compared across multiple losses.
• The combination of metric-learning and classification-based losses generally yields best performance across scenarios.
• Embedding BN provides significant gains for classification-based objectives.
• The best single model (AP+Softmax with H/ASP) achieved competitive results, and the reported best performance for the VoxSRC 2020 test set was an EER of 5.19% and MinDCF of 0.314.
• The authors demonstrated strong results without relying on model ensembles or post-processing and released training code and pretrained baselines for the community.