[Paper Review] CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordings

Shinji Watanabe, Michael Mandel+19 more|arXiv (Cornell University)|Apr 20, 2020

Speech Recognition and SynthesisComputer Science49 references97 citations

TL;DR

The paper introduces CHiME-6 with two tracks: Track 1 for segmented multispeaker ASR and Track 2 for unsegmented multispeaker ASR with diarization, providing open-source Kaldi baselines for end-to-end multispeaker processing.

ABSTRACT

Following the success of the 1st, 2nd, 3rd, 4th and 5th CHiME challenges we organize the 6th CHiME Speech Separation and Recognition Challenge (CHiME-6). The new challenge revisits the previous CHiME-5 challenge and further considers the problem of distant multi-microphone conversational speech diarization and recognition in everyday home environments. Speech material is the same as the previous CHiME-5 recordings except for accurate array synchronization. The material was elicited using a dinner party scenario with efforts taken to capture data that is representative of natural conversational speech. This paper provides a baseline description of the CHiME-6 challenge for both segmented multispeaker speech recognition (Track 1) and unsegmented multispeaker speech recognition (Track 2). Of note, Track 2 is the first challenge activity in the community to tackle an unsegmented multispeaker speech recognition scenario with a complete set of reproducible open source baselines providing speech enhancement, speaker diarization, and speech recognition modules.

Motivation & Objective

• Advance distant-microphone multispeaker ASR in real home settings through two tracks (segmented and unsegmented).
• Provide reproducible baselines including speech enhancement, diarization, and ASR components integrated in Kaldi.
• Quantify the impact of diarization errors on recognition performance in a realistic, diarization-enabled setting.
• Offer open-source recipes to lower entry barriers for researchers tackling unsegmented multispeaker ASR.

Proposed method

• Two challenge tracks: Track 1 (ASR with ground-truth diarization) and Track 2 (diarization + ASR).
• Array synchronization baseline to align multiple commercial 4-channel microphone arrays.
• Speech enhancement front-ends based on guided source separation (GSS) and BeamformIt with optional WPE dereverberation.
• Kaldi-based ASR pipelines including MFCC features, GMM-HMM and chain TDNN-F acoustic models.
• Data augmentation, data preparation, and decoding with two-stage i-vector/beamforming-enhanced decoding.
• Diarization pipeline in Track 2 using x-vectors (TDNN) with PLDA scoring and AHC, plus RTTM-based evaluation.

Experimental results

Research questions

• RQ1 How does diarization impact ASR performance in unsegmented multispeaker recordings?
• RQ2 Can reproducible open-source baselines for synchronization, enhancement, diarization, and ASR simplify entry into CHiME-6-style tasks?
• RQ3 What are the baseline performance gaps between segmented ASR and unsegmented multispeaker ASR in realistic home environments?

Key findings

• Track 1 baseline ASR WER: Dev 51.8%, Eval 51.3%.
• Track 2 baseline SAD results (Annotation RTTM): DEV DER 61.6%, JER 69.8%; EVAL DER 62.0%, JER 71.4%.
• Track 2 baseline SAD results (Alignment RTTM): DEV DER 63.4%, JER 70.8%; EVAL DER 68.2%, JER 72.5%.
• Track 1 enhancement with BeamformIt: WER 69.8% (DEV) and 61.2% (EVAL).
• Track 1 enhancement with GSS: WER 51.8% (DEV) and 51.3% (EVAL).
• Track 2 enhancement with BeamformIt: WER 84.3% (DEV) and 77.9% (EVAL).