[Paper Review] Noise Robust IOA/CAS Speech Separation and Recognition System For The Third 'CHIME' Challenge

Xiaofei Wang, Chao Wu+6 more|arXiv (Cornell University)|Sep 21, 2015

Speech and Audio Processing8 references6 citations

TL;DR

This paper presents a noise-robust speech separation and recognition system for the third CHiME challenge, combining a speech distortion-weighted multi-channel Wiener filter (SDW-MWF) for front-end enhancement with deep neural networks (DNN, CNN, LSTM) and lattice rescoring for back-end ASR. The system achieves a 60.9% relative word error rate (WER) reduction, reaching 13.2% WER on real test data, significantly outperforming the best GMM baseline of 33.23%.

ABSTRACT

This paper presents the contribution to the third 'CHiME' speech separation and recognition challenge including both front-end signal processing and back-end speech recognition. In the front-end, Multi-channel Wiener filter (MWF) is designed to achieve background noise reduction. Different from traditional MWF, optimized parameter for the tradeoff between noise reduction and target signal distortion is built according to the desired noise reduction level. In the back-end, several techniques are taken advantage to improve the noisy Automatic Speech Recognition (ASR) performance including Deep Neural Network (DNN), Convolutional Neural Network (CNN) and Long short-term memory (LSTM) using medium vocabulary, Lattice rescoring with a big vocabulary language model finite state transducer, and ROVER scheme. Experimental results show the proposed system combining front-end and back-end is effective to improve the ASR performance.

Motivation & Objective

• To improve automatic speech recognition (ASR) performance in noisy, real-world environments such as restaurants, buses, and streets.
• To address the trade-off between noise reduction and speech distortion in multi-channel speech enhancement using an optimized front-end.
• To enhance back-end ASR robustness through deep learning models (DNN, CNN, LSTM) and lattice rescoring with large-vocabulary language models.
• To achieve state-of-the-art performance on the third CHiME speech separation and recognition challenge under realistic noisy conditions.

Proposed method

• Employed a speech distortion-weighted multi-channel Wiener filter (SDW-MWF) to estimate the desired speech signal, optimizing the trade-off between noise reduction and speech distortion via a tunable parameter μ.
• Derived the SDW-MWF filter using the optimization criterion: minimize the expected value of speech distortion plus weighted noise power, formulated as min E{|w^H y - X1|^2 + μ|w^H n|^2}.
• Applied the SDW-MWF to both training and test data to reduce mismatch between training and test conditions, using random SNR perturbations (-6dB to +6dB) for data augmentation.
• Used DNN, CNN, and LSTM acoustic models for end-to-end acoustic modeling, with DNN and CNN showing superior performance over GMM baselines.
• Implemented lattice rescoring using a large-vocabulary finite-state transducer language model to refine hypothesis outputs.
• Combined multiple system outputs via the ROVER (Recognition Output Voting Error Reduction) scheme to improve final recognition accuracy.

Experimental results

Research questions

• RQ1 How can the trade-off between noise reduction and speech distortion be optimized in multi-channel speech enhancement for robust ASR?
• RQ2 To what extent do deep neural networks (DNN, CNN, LSTM) improve ASR performance in noisy, reverberant environments compared to traditional GMM-HMM systems?
• RQ3 Can lattice rescoring with a large-vocabulary language model further reduce word error rates in noisy speech recognition?
• RQ4 How does combining multiple neural network-based systems via ROVER improve overall ASR performance on real-world test sets?

Key findings

• The SDW-MWF front-end reduced WER from 37.36% (MVDR baseline) to 23.19% on real test data under the GMM model, demonstrating significant noise suppression with controlled distortion.
• Using DNN with sMBR training and random SNR data augmentation reduced WER to 18.4% on real test data, showing the effectiveness of deep learning in noisy conditions.
• The CNN-based acoustic model further reduced WER to 17.87%, and LSTM achieved 18.96% WER, indicating that recurrent and convolutional networks better model temporal and spectral features.
• Lattice rescoring reduced WER from 18.4% to 15.3% (DNN+sMBR) and from 17.87% to 16.37% (CNN+sMBR), proving its effectiveness in refining ASR outputs.
• The final system using ROVER fusion of DNN, CNN, and LSTM systems with lattice rescoring achieved a WER of 13.2% on real test data, representing a 60.9% relative reduction compared to the best GMM baseline (33.23%).
• The best single system (DNN+sMBR with lattice rescoring) achieved 17.74% WER on the bus scenario, 11.75% on the café, 13.34% on the pedestrian area, and 9.96% on the street junction, showing strong robustness across environments.