[Paper Review] Deep Speech Enhancement for Reverberated and Noisy Signals using Wide Residual Networks

Dayana Ribas, Jorge Llombart+2 more|arXiv (Cornell University)|Jan 3, 2019

Speech and Audio Processing58 references15 citations

TL;DR

This paper proposes a deep speech enhancement method using Wide Residual Networks (WRN) with 1D convolutions to model temporal dependencies in reverberated and noisy speech. The residual connections enable effective learning of clean speech reconstruction, significantly improving both speech quality and automatic speech recognition (ASR) performance, especially in low-SNR and far-field conditions, outperforming traditional methods like MM-LSA.

ABSTRACT

This paper proposes a deep speech enhancement method which exploits the high potential of residual connections in a wide neural network architecture, a topology known as Wide Residual Network. This is supported on single dimensional convolutions computed alongside the time domain, which is a powerful approach to process contextually correlated representations through the temporal domain, such as speech feature sequences. We find the residual mechanism extremely useful for the enhancement task since the signal always has a linear shortcut and the non-linear path enhances it in several steps by adding or subtracting corrections. The enhancement capacity of the proposal is assessed by objective quality metrics and the performance of a speech recognition system. This was evaluated in the framework of the REVERB Challenge dataset, including simulated and real samples of reverberated and noisy speech signals. Results showed that enhanced speech from the proposed method succeeded for both, the enhancement task with intelligibility purposes and the speech recognition system. The DNN model, trained with artificial synthesized reverberation data, was able to deal with far-field reverberated speech from real scenarios. Furthermore, the method was able to take advantage of the residual connection achieving to enhance signals with low noise level, which is usually a strong handicap of traditional enhancement methods.

Motivation & Objective

• To address the limitations of traditional statistical speech enhancement methods, which struggle with non-stationary noise and reverberation.
• To improve speech intelligibility and robustness for downstream ASR systems in real-world acoustic conditions.
• To leverage deep learning, specifically Wide Residual Networks (WRN), to model complex temporal and spectral correlations in speech signals.
• To overcome the common issue of performance degradation in low-SNR scenarios, where traditional methods often fail.
• To demonstrate that residual learning in wide CNN architectures enables effective enhancement of both noisy and reverberated speech.

Proposed method

• The method employs a Wide Residual Network (WRN) architecture with 1D convolutions applied along the time dimension to capture temporal context in speech feature sequences.
• Residual connections are used to stabilize training and allow learning of identity mappings, where the skip connection preserves the original signal and the non-linear path adds learned corrections.
• The network is trained end-to-end on artificially synthesized reverberation data to learn the mapping from noisy, reverberated speech to clean speech.
• The model uses a multi-scale receptive field via dilated convolutions to capture long-range dependencies in the temporal domain.
• The loss function is based on mean squared error (MSE) between predicted and target clean speech features, optimized via backpropagation.
• The approach is evaluated using objective speech quality metrics (LLR, CD) and automatic speech recognition (ASR) word error rate (WER) on the REVERB Challenge dataset.

Experimental results

Research questions

• RQ1 Can a Wide Residual Network architecture effectively enhance reverberated and noisy speech signals while preserving intelligibility?
• RQ2 How does the residual learning mechanism in WRN improve performance compared to traditional statistical methods like MM-LSA?
• RQ3 To what extent can a model trained on synthetic reverberation generalize to real-world far-field, reverberant speech conditions?
• RQ4 Does the proposed method maintain or improve performance in low signal-to-noise ratio (SNR) conditions, where traditional methods often fail?
• RQ5 Can the WRN-based enhancement improve downstream ASR performance across diverse acoustic environments, including simulated and real recordings?

Key findings

• The WRN-based enhancement method significantly outperformed the MM-LSA baseline in both speech quality metrics (LLR and CD) and ASR word error rate (WER) across all test conditions.
• On the REVERB Development dataset, the WRN model achieved a WER of 45.89% (GMM-HMM) and 47.29% (GMM-HMM MCT + fMLLR) for far-field reverberated speech, compared to 89.59% for the baseline noisy input.
• For the REVERB Evaluation dataset, the WRN model achieved a WER of 44.14% (GMM-HMM) and 46.85% (GMM-HMM MCT + fMLLR), compared to 88.41% for the baseline noisy input.
• The model demonstrated strong generalization to real-world far-field scenarios, even when trained only on synthetic data with artificial reverberation.
• The method successfully enhanced low-SNR speech, a known challenge for traditional enhancement techniques, indicating the effectiveness of residual learning in preserving clean signal components.
• The residual mechanism enabled stable training and effective signal reconstruction by allowing the network to learn residual corrections to a linear shortcut of the input signal.