[Paper Review] A Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech Enhancement

Jean-Marc Valin|arXiv (Cornell University)|Sep 24, 2017

Speech and Audio Processing4 citations

TL;DR

This paper presents a hybrid DSP/deep learning approach for real-time full-band (48 kHz) speech enhancement that uses a 4-layer recurrent neural network to estimate ideal critical band gains for 22 Bark-scale bands, combined with a pitch comb filter to suppress noise between harmonics. The method achieves significantly higher speech quality than traditional MMSE-based methods while maintaining low computational complexity suitable for real-time deployment on low-power CPUs.

ABSTRACT

Despite noise suppression being a mature area in signal processing, it remains highly dependent on fine tuning of estimator algorithms and parameters. In this paper, we demonstrate a hybrid DSP/deep learning approach to noise suppression. A deep neural network with four hidden layers is used to estimate ideal critical band gains, while a more traditional pitch filter attenuates noise between pitch harmonics. The approach achieves significantly higher quality than a traditional minimum mean squared error spectral estimator, while keeping the complexity low enough for real-time operation at 48 kHz on a low-power processor.

Motivation & Objective

• To address the persistent challenge of tuning traditional noise suppression algorithms, which rely on complex, manually tuned estimators for noise power and spectral gains.
• To reduce computational complexity in full-band speech enhancement while maintaining high-quality output suitable for real-time applications like video conferencing.
• To combine the robustness of established DSP techniques with the adaptive learning capability of deep learning, focusing on components that are difficult to tune manually.
• To demonstrate that a lightweight deep learning model can significantly improve speech quality over conventional MMSE-based methods without requiring high-end hardware.

Proposed method

• A 4-layer recurrent neural network (RNN) estimates ideal critical band gains (ICBGs) for 22 Bark-scale bands, with outputs bounded between 0 and 1, reducing the need for high-dimensional spectral estimation.
• The network processes 20 ms frames with 50% overlap using a Vorbis window that satisfies the Princen-Bradley criterion for perfect reconstruction.
• Per-band gains are applied via a weighted interpolation across frequency bins using triangular band weights, ensuring smooth spectral shaping.
• A pitch comb filter is applied in the frequency domain using band-specific coefficients to suppress noise between pitch harmonics, based on periodicity estimates.
• Gain smoothing is applied via a recursive filter with a decay factor λ=0.6 to prevent over-drying the output and preserve natural reverberation.
• The model is trained using a combination of mean squared error for ICBG estimation and cross-entropy for voice activity detection (VAD), with masked loss for low-energy frames.

Experimental results

Research questions

• RQ1 Can a lightweight deep learning model effectively replace the most difficult-to-tune components in a traditional noise suppression pipeline?
• RQ2 Does combining a low-resolution deep learning gain estimator with a pitch-based filter outperform conventional MMSE-based spectral estimators in full-band speech enhancement?
• RQ3 Can such a hybrid system achieve high-quality speech enhancement at low computational complexity suitable for real-time operation on low-power CPUs?
• RQ4 To what extent does the proposed method improve speech quality over traditional DSP-only approaches, especially for non-stationary noise types?

Key findings

• The proposed method achieves significantly higher speech quality than the MMSE-based noise suppressor in SpeexDSP, as confirmed by PESQ MOS-LQO scores across babble, car, and street noise conditions.
• The system operates in real time at 48 kHz on a low-power CPU, requiring only 1.3% of a single x86 core (Haswell i7-4800MQ) and 14% on a Raspberry Pi 3’s ARM Cortex-A53 core.
• Total computational complexity is approximately 40 Mflops, comparable to a full-band speech coder, and the model’s 87,503 weights can be quantized to 8-bit without performance loss.
• The use of gain smoothing with λ=0.6 effectively prevents over-drying the output, preserving natural reverberation with a 135 ms effective decay time.
• The approach is extensible to residual echo suppression and microphone array post-filtering by augmenting input features with far-end signal characteristics.