[Paper Review] Single Channel Audio Source Separation using Convolutional Denoising Autoencoders

Emad M. Grais, Mark D. Plumbley|arXiv (Cornell University)|Mar 23, 2017

Speech and Audio Processing27 references14 citations

TL;DR

This paper proposes using deep fully convolutional denoising autoencoders (CDAEs) for single-channel audio source separation, training one CDAE per source to extract target components while treating others as noise. Experiments show CDAEs achieve competitive performance with fewer parameters than feedforward neural networks, especially excelling in separating challenging sources like drums.

ABSTRACT

Deep learning techniques have been used recently to tackle the audio source separation problem. In this work, we propose to use deep fully convolutional denoising autoencoders (CDAEs) for monaural audio source separation. We use as many CDAEs as the number of sources to be separated from the mixed signal. Each CDAE is trained to separate one source and treats the other sources as background noise. The main idea is to allow each CDAE to learn suitable spectral-temporal filters and features to its corresponding source. Our experimental results show that CDAEs perform source separation slightly better than the deep feedforward neural networks (FNNs) even with fewer parameters than FNNs.

Motivation & Objective

• To address the challenge of single-channel audio source separation (SCSS) using deep learning.
• To explore whether convolutional denoising autoencoders (CDAEs) can learn robust spectral-temporal features for source separation.
• To compare CDAE performance against fully connected feedforward neural networks (FNNs) in terms of separation quality and parameter efficiency.
• To evaluate the effectiveness of CDAEs in handling highly nonstationary background noise during source separation.

Proposed method

• Each source is separated using a dedicated CDAE that treats the other sources as background noise.
• The CDAE architecture uses 2D convolutional layers, ReLU activation, max-pooling in the encoder, and transposed convolutions with up-sampling in the decoder.
• Input to each CDAE is a 2D magnitude spectrogram segment of the mixed signal, with noise introduced during training to enhance robustness.
• Spectral masks are computed from the CDAE outputs using the formula $ \mathbf{M}_{i}(n,f) = \frac{\mathbf{\tilde{S}}_{i}(n,f)}{\sum_{j}^{I}\mathbf{\tilde{S}}_{j}(n,f)} $ to ensure energy conservation.
• The final source estimate is obtained via inverse STFT using the phase of the mixed signal and the masked magnitude spectrogram.
• Training uses Nesterov’s accelerated gradient with early stopping and learning rate decay, optimized on validation loss.

Experimental results

Research questions

• RQ1 Can fully convolutional denoising autoencoders (CDAEs) effectively separate individual sources from a single-channel audio mixture?
• RQ2 How does the performance of CDAEs compare to fully connected feedforward neural networks (FNNs) in terms of separation quality and parameter efficiency?
• RQ3 Does the CDAE architecture better capture spectral-temporal structures in audio signals than fully connected networks?
• RQ4 How well do CDAEs perform on difficult-to-separate sources such as drums?
• RQ5 Can CDAEs achieve high-quality separation with fewer parameters than FNNs?

Key findings

• CDAEs achieve slightly better source separation performance than FNNs despite having fewer parameters.
• CDAEs show significant improvement over FNNs in separating drums, a particularly challenging source.
• Normalized SDR and SIR values are positive for CDAEs, indicating effective separation and noise suppression.
• CDAEs outperform FNNs in SAR for all sources except bass, indicating fewer artifacts in the separated signals.
• The performance of CDAEs and FNNs is comparable for vocals and bass in terms of SDR and SIR.
• The use of spectral masking ensures that the sum of estimated sources approximates the original mixed signal.