[Paper Review] Nonlinear Residual Echo Suppression Based on Multi-stream Conv-TasNet

Hongsheng Chen, Teng Xiang+2 more|arXiv (Cornell University)|May 15, 2020

Speech and Audio Processing23 references4 citations

TL;DR

This paper proposes a multi-stream Conv-TasNet-based residual echo suppression (RES) method that leverages both the linear AEC system's residual signal and the adaptive filter's output as dual input streams to enhance echo suppression. By integrating multi-input convolutional blocks and an exponential layer normalization, the model achieves superior performance in both single-talk and double-talk scenarios, with over 45 dB ERLE and SDR gains exceeding 13 dB in double-talk conditions.

ABSTRACT

Acoustic echo cannot be entirely removed by linear adaptive filters due to the nonlinear relationship between the echo and far-end signal. Usually a post processing module is required to further suppress the echo. In this paper, we propose a residual echo suppression method based on the modification of fully convolutional time-domain audio separation network (Conv-TasNet). Both the residual signal of the linear acoustic echo cancellation system, and the output of the adaptive filter are adopted to form multiple streams for the Conv-TasNet, resulting in more effective echo suppression while keeping a lower latency of the whole system. Simulation results validate the efficacy of the proposed method in both single-talk and double-talk situations.

Motivation & Objective

• Address the limitation of linear acoustic echo cancellation (LAEC) in handling nonlinear echo paths.
• Overcome the trade-off between residual echo suppression and near-end speech distortion in traditional RES methods.
• Leverage deep learning to model complex nonlinear relationships in residual echo suppression.
• Improve speech quality and echo suppression by exploiting auxiliary information from the adaptive filter output.
• Develop a low-latency, end-to-end time-domain solution that outperforms frequency-domain and recurrent network-based approaches.

Proposed method

• Adapt the Conv-TasNet architecture to process two input streams: the LAEC residual signal $s_{\text{AEC}}(n)$ and the adaptive filter output $\hat{d}(n)$.
• Introduce multi-input convolutional (MI Conv) blocks in the temporal convolutional network (TCN) to jointly model features from both input streams.
• Replace cumulative layer normalization (cLN) with exponential layer normalization (eLN) to stabilize training and improve generalization.
• Modify the training objective to emphasize near-end speech recovery and suppress residual echo, breaking the channel balance in standard speech separation.
• Use a 1D convolutional encoder to map waveforms to embeddings, followed by a TCN-based suppression module and a transposed convolutional decoder for waveform reconstruction.
• Apply end-to-end training with a loss function optimized for speech quality and echo suppression, avoiding phase reconstruction issues inherent in STFT-based methods.

Experimental results

Research questions

• RQ1 Can combining the LAEC residual signal and adaptive filter output as dual streams improve residual echo suppression performance compared to single-stream models?
• RQ2 Does the integration of multi-input convolutional blocks enhance the model's ability to suppress nonlinear residual echoes while preserving near-end speech quality?
• RQ3 How does the proposed method compare to conventional RES methods (e.g., FCN, BLSTM) in terms of echo attenuation and speech quality metrics?
• RQ4 To what extent does the proposed method reduce latency compared to non-causal RNN-based approaches like BLSTM?
• RQ5 Can the modified Conv-TasNet architecture achieve better performance than standard Conv-TasNet in double-talk scenarios with high echo-to-speech energy ratios?

Key findings

• The proposed multi-stream Conv-TasNet (TasNet-MI) achieves over 45 dB echo return loss enhancement (ERLE) in single-talk scenarios for both speech and music, significantly outperforming FCN (16.55 dB) and BLSTM (51.67 dB for speech).
• In double-talk conditions with -14.2 dB SER, the TasNet-MI achieves a PESQ of 2.80 and SDR of 13.8 dB for speech, outperforming TasNet-L (PESQ: 2.71, SDR: 12.6 dB) and TasNet-O (PESQ: 2.57, SDR: 11.7 dB).
• For music far-end signals at -18.2 dB SER, the TasNet-MI achieves STOI of 0.820, surpassing TasNet-L (0.800) and TasNet-O (0.773), indicating improved speech intelligibility.
• The model maintains low latency due to its fully convolutional, causal architecture, avoiding the high delay of non-causal BLSTM-based methods.
• The addition of MI Conv blocks and eLN leads to a 1.1 dB SDR improvement over TasNet-O and a 2.2 dB SDR gain over TasNet-L in double-talk speech conditions.
• The method achieves a 13.8 dB SDR gain over LAEC-only processing in double-talk speech conditions, demonstrating effective echo suppression without degrading near-end speech quality.