[Paper Review] USTC-NELSLIP System Description for DIHARD-III Challenge

Yuxuan Wang, Maokui He+7 more|arXiv (Cornell University)|Mar 19, 2021

Speech Recognition and SynthesisComputer Science21 references20 citations

TL;DR

This paper presents the USTC-NELSLIP system for the DIHARD-III Speech Diarization Challenge, introducing iterative speech separation (ISS) and iterative target-speaker VAD (ITS-VAD) to enhance diarization in noisy, multi-speaker environments. By combining domain-dependent processing, audio domain classification, and model fusion, the system achieved a 11.30% DER on track 1 and 16.78% on track 2, ranking first on the leaderboard with a 48.7% relative improvement over the baseline clustering system.

ABSTRACT

This system description describes our submission system to the Third DIHARD Speech Diarization Challenge. Besides the traditional clustering based system, the innovation of our system lies in the combination of various front-end techniques to solve the diarization problem, including speech separation and target-speaker based voice activity detection (TS-VAD), combined with iterative data purification. We also adopted audio domain classification to design domain-dependent processing. Finally, we performed post processing to do system fusion and selection. Our best system achieved DERs of 11.30% in track 1 and 16.78% in track 2 on evaluation set, respectively.

Motivation & Objective

• To improve speech diarization performance in challenging, real-world environments with overlapping speech and background noise.
• To address domain-specific variations in diarization performance through domain classification and tailored processing pipelines.
• To enhance diarization accuracy by integrating iterative speech separation (ISS) and iterative target-speaker VAD (ITS-VAD) systems.
• To reduce diarization error rate (DER) through multi-system fusion and iterative data purification.
• To achieve state-of-the-art performance on the DIHARD-III challenge using end-to-end and hybrid modular approaches.

Proposed method

• A 17-layer ResNet-based audio domain classifier was trained on 10-second segments to identify 11 distinct audio domains, achieving 100% accuracy on the development set.
• Progressive multi-target speech enhancement was applied to the RESTAURANT domain using a 10 dB SNR increment per layer, with PELPS and PRM features used for enhancement.
• An iterative speech separation (ISS) system was trained on 250 hours of simulated 2-speaker Librispeech data to improve speaker separation in overlapping speech.
• An iterative target-speaker VAD (ITS-VAD) system used i-vector extraction on augmented VoxCeleb data and was trained on 2500 hours of realistic and simulated multi-speaker data.
• Multiple SAD models (DNN, CLDNN, TDNN) were trained and fused via weighted voting to improve speech activity detection accuracy.
• System fusion was performed using dover-lap on multiple iterations of ISS and ITS-VAD outputs, with final results selected via post-processing and domain-specific routing.

Experimental results

Research questions

• RQ1 Can iterative speech separation significantly improve diarization performance in overlapping, noisy speech environments?
• RQ2 How does iterative target-speaker VAD compare to traditional clustering-based diarization across diverse audio domains?
• RQ3 To what extent does domain-dependent processing enhance diarization robustness across heterogeneous real-world recordings?
• RQ4 What is the impact of combining speech enhancement, SAD fusion, and iterative refinement on overall diarization error rate?
• RQ5 Can model fusion via dover-lap further reduce DER when combining multiple iterative diarization systems?

Key findings

• The USTC-NELSLIP system achieved a 11.30% diarization error rate (DER) on the track 1 evaluation set, representing a 48.7% relative improvement over the baseline clustering system.
• On track 2, the system achieved a DER of 16.78%, significantly outperforming the baseline system which had a DER of 25.36% on the full evaluation set.
• The integration of the ISS-based system reduced DER from 16.22% to 8.31% when combined with iterative training and dover-lap fusion, demonstrating strong iterative refinement gains.
• The fusion of three SAD models (DNN, CLDNN, TDNN) reduced the overall SAD error from 2.42% (baseline) to 1.36% on the full development set, showing effective complementarity.
• The ITS-VAD system outperformed the baseline clustering system across all domains except RESTAURANT, where performance degraded due to high noise and overlap, justifying domain-specific routing.
• Audio domain classification achieved 100% accuracy on the development set, enabling effective domain-aware processing and routing of audio streams.