[Paper Review] VoxSRC 2019: The first VoxCeleb Speaker Recognition Challenge

Joon Son Chung, Arsha Nagrani+5 more|arXiv (Cornell University)|Dec 5, 2019

Speech Recognition and SynthesisComputer Science19 references48 citations

TL;DR

The paper introduces VoxSRC 2019, the first public VoxCeleb-based speaker verification challenge with fixed/open training tracks, detailing data, evaluation (EER), baselines, results, and lessons.

ABSTRACT

The VoxCeleb Speaker Recognition Challenge 2019 aimed to assess how well current speaker recognition technology is able to identify speakers in unconstrained or `in the wild' data. It consisted of: (i) a publicly available speaker recognition dataset from YouTube videos together with ground truth annotation and standardised evaluation software; and (ii) a public challenge and workshop held at Interspeech 2019 in Graz, Austria. This paper outlines the challenge and provides its baselines, results and discussions.

Motivation & Objective

• Motivate evaluation of speaker verification under unconstrained, 'in the wild' conditions using a public dataset and challenge framework.
• Provide standardized data, evaluation software, and a workshop to foster progress in robust speaker recognition.
• Present baseline systems, challenge results, and insights to guide future research under realistic conditions.

Proposed method

• Two tracks define fixed and open training conditions for speaker verification tasks.
• Public VoxCeleb-derived data from YouTube with ground truth and validation/test splits.
• Real-valued similarity scoring with EER as the primary evaluation metric.
• A simple CNN-based baseline trained on VoxCeleb2 development data.
• Winning systems use deep neural networks (TDNNs, ResNets) with front-end embedding extraction and back-end scoring (G-PLDA, cosine), with data augmentation and regularization.

Experimental results

Research questions

• RQ1 How well do current speaker verification methods perform on unconstrained, in-the-wild data under fixed and open training conditions?
• RQ2 What data, augmentation, and model design choices yield the best EER in VoxSRC 2019?
• RQ3 To what extent does additional training data help in the open condition given potential domain transfer issues?

Key findings

• Over 50 teams participated; 90% beat the provided baseline in the fixed condition and 85% beat it in the open condition.
• Top methods rely on embedding extraction networks (TDNNs, ResNets) plus back-end classifiers (G-PLDA, cosine) with score normalization and ensemble fusion.
• Heavy data augmentation and regularization (e.g., RIR, Musan noise, margin-based losses) were effective.
• Open-condition best EER (0.0126) was only marginally better than fixed-condition best (0.0142), suggesting domain transfer limits from extra data.
• Phonetic attention and SAD had limited impact on VoxSRC due to mostly non-silent utterances.