[Paper Review] N-HANS: Introducing the Augsburg Neuro-Holistic Audio-eNhancement System

Shuo Liu, Gil Keren+1 more|arXiv (Cornell University)|Nov 16, 2019

Speech and Audio Processing16 references4 citations

TL;DR

N-HANS is a TensorFlow-based Python toolkit for in-the-wild audio enhancement, enabling denoising, source separation, and selective noise suppression via a dual-branch ±Auxiliary Network that conditions on positive and negative reference recordings. It achieves state-of-the-art performance, with SDR up to 11.86 dB and PESQ of 0.92 on LibriSpeech and AudioSet benchmarks, demonstrating robustness across diverse real-world conditions.

ABSTRACT

N-HANS is a Python toolkit for in-the-wild audio enhancement, including speech, music, and general audio denoising, separation, and selective noise or source suppression. The functionalities are realised based on two neural network models sharing the same architecture, but trained separately. The models are comprised of stacks of residual blocks, each conditioned on additional speech or environmental noise recordings for adapting to different unseen speakers or environments in real life. In addition to a Python API, a command line interface is provided to researchers and developers, both of which are documented at https://github.com/N-HANS/N-HANS. Experimental results indicate that N-HANS achieves outstanding performance, and ensure its reliable usage in real-life audio and speech-related tasks, reaching very high audio and speech quality.

Motivation & Objective

• To address the challenge of enhancing real-world audio corrupted by unseen noise and interfering sources, especially in low-SNR and complex acoustic environments.
• To develop a unified system capable of denoising, source separation, and selective noise suppression using a single architecture with shared components.
• To improve audio quality and intelligibility while preserving desired signals such as speech or environmental warnings.
• To enable adaptation to unseen speakers and environments through reference-based conditioning, enhancing generalization beyond training data.

Proposed method

• The system employs two identical residual neural network architectures, one for positive and one for negative reference conditioning, forming the ±Auxiliary Network.
• Log magnitude STFTs of noisy, positive, and negative reference audio are fed into separate branches to generate embedding vectors for positive and negative components.
• The enhancement network combines the noisy audio input with the positive and negative embeddings to produce a refined output signal.
• The model is trained end-to-end using a combination of perceptual and objective loss functions to preserve speech naturalness and minimize distortion.
• A command-line interface and Python API are provided for integration into speech and audio processing pipelines.
• The system supports plug-in integration with openXBOW and auDeep for audio feature extraction.

Experimental results

Research questions

• RQ1 Can a single neural network architecture effectively handle multiple audio enhancement tasks—denoising, separation, and selective suppression—using reference conditioning?
• RQ2 How well can the model generalize to unseen speakers and environmental noise conditions using only reference recordings?
• RQ3 To what extent does reference-based conditioning improve performance in low-SNR and complex acoustic scenarios compared to fixed-model baselines?
• RQ4 Can the system preserve useful non-speech signals (e.g., alarms) while suppressing only targeted noise or interference?
• RQ5 What is the impact of varying signal-to-noise ratios on the perceptual and objective quality of enhanced audio?

Key findings

• On the LibriSpeech and AudioSet test set, N-HANS achieved a signal-to-distortion ratio (SDR) of 11.86 dB at 10 dB input SNR, indicating strong noise suppression capability.
• For selective denoising with 8 dB positive and negative noise, the system reached a PESQ score of 0.90 and an SDR of 12.12 dB, demonstrating high speech quality and robustness.
• In speech separation on VoxCeleb, N-HANS achieved an SDR of 4.79 dB across all mixtures, outperforming Deep Clustering (0.84 dB) and Deep Attractor (1.81 dB).
• The system maintained high intelligibility, with STOI scores reaching 0.90 under high SNR conditions, indicating preserved speech clarity.
• Mel cepstral distortion (MCD) was reduced to 5.98 at 10 dB SNR, indicating minimal spectral distortion and high signal fidelity.
• The system demonstrated consistent performance across diverse noise types and speaker combinations, confirming its generalization to unseen environments.