[Paper Review] W-Net BF: DNN-based Beamformer Using Joint Training Approach

Yuichiro Koyama, Bhiksha Raj|arXiv (Cornell University)|Oct 31, 2019

Speech and Audio Processing28 references4 citations

TL;DR

This paper proposes W-Net BF, a DNN-based beamformer that jointly trains a noise-masked reference estimator and a beamforming filter estimator to combine the generalization of mask-based methods with the filter-optimality of direct filter estimation. Evaluated on static and moving noise sources, W-Net BF outperforms both U-Net BF and BLSTM-GEV across SNR, SDR, STOI, and PESQ metrics, demonstrating superior robustness to spatially non-stationary noise.

ABSTRACT

Acoustic beamformers have been widely used to enhance audio signals. The best current methods are DNN-powered variants of the generalized eigenvalue beamformer, and DNN-based filterestimation methods that directly compute beamforming filters. Both approaches, while effective, have blindspots in their generalizability. We propose a novel approach that combines both approaches into a single framework that attempts to exploit the best features of both. The resulting model, called a W-Net beamformer, includes two components: the first computes a noise-masked reference which the second uses to estimate beamforming filters. Results on data that include a wide variety of room and noise conditions, including static and mobile noise sources, show that the proposed beamformer outperforms other methods in all tested evaluation metrics.

Motivation & Objective

• To address the generalization limitations of existing DNN-based beamformers that rely solely on mask estimation or direct filter estimation.
• To develop a unified framework that leverages the robustness of mask-based methods and the filtering efficiency of filter-estimation approaches.
• To improve performance in non-stationary acoustic environments, particularly with moving noise sources.
• To validate the effectiveness of joint training in enhancing beamformer generalization and robustness.

Proposed method

• The W-Net BF architecture consists of two stages: a reference estimation module that generates a noise-masked spectrogram using a DNN, and a beamforming filter estimation module that uses this reference to compute optimal filters.
• The model employs joint end-to-end training to optimize both modules simultaneously, enabling better alignment between reference estimation and filter learning.
• The reference estimation module uses a U-Net-like encoder-decoder structure to predict time-frequency masks for noise and speech components.
• The filter estimation module uses a fully convolutional network to predict complex-valued beamforming filters in the time-frequency domain.
• The loss function combines SNR maximization and mask prediction objectives, enabling joint optimization of both components.
• The framework is trained on simulated data with diverse room impulse responses, speech sources, and static/moving noise sources, with fine-tuning for moving noise scenarios.

Experimental results

Research questions

• RQ1 Can a joint training framework improve beamformer performance by combining the strengths of mask-based and filter-estimation-based DNN beamformers?
• RQ2 Does the proposed W-Net BF architecture generalize better than standalone mask-based or filter-estimation models in non-stationary noise environments?
• RQ3 How does joint optimization of reference estimation and filter learning affect robustness to moving noise sources?
• RQ4 To what extent does the W-Net BF outperform existing state-of-the-art beamformers in terms of SNR, SDR, STOI, and PESQ?

Key findings

• W-Net BF achieves a mean SNR of 14.2 dB on the Static-Dataset, outperforming U-Net BF (13.5 dB) and BLSTM-GEV (12.8 dB).
• On the Moving-Dataset, W-Net BF achieves 13.0 dB SNR, significantly outperforming BLSTM-GEV (11.5 dB), demonstrating superior handling of moving noise sources.
• W-Net BF achieves a mean SDR of 10.1 dB on Static-Dataset, surpassing U-Net BF (9.3 dB) and BLSTM-GEV (8.6 dB), indicating lower distortion in enhanced signals.
• The joint training component (W-Net BF vs. W-Net BF†) improves performance by 0.7 dB in SNR and 0.8 dB in SDR, confirming the benefit of end-to-end optimization.
• W-Net BF-m, fine-tuned on moving noise data, achieves 12.5 dB SNR on Moving-Dataset, showing that joint learning enables adaptation to dynamic noise conditions.
• The model achieves a STOI score of 0.85 and PESQ of 2.7 on Static-Dataset, indicating high speech intelligibility and perceptual quality.