[Paper Review] End-to-End Multi-Task Learning for Adjustable Joint Noise Reduction and Hearing Loss Compensation

Philippe Gonzalez, Vera Margrethe Frederiksen+2 more|arXiv (Cornell University)|Mar 20, 2026

Hearing Loss and Rehabilitation0 citations

TL;DR

The authors propose an end-to-end multi-task DNN that jointly performs noise reduction and hearing loss compensation with independently adjustable amounts at inference, personalized via audiogram input and trained with a differentiable auditory model.

ABSTRACT

A multi-task learning framework is proposed for optimizing a single deep neural network (DNN) for joint noise reduction (NR) and hearing loss compensation (HLC). A distinct training objective is defined for each task, and the DNN predicts two time-frequency masks. During inference, the amounts of NR and HLC can be adjusted independently by exponentiating each mask before combining them. In contrast to recent approaches that rely on training an auditory-model emulator to define a differentiable training objective, we propose an auditory model that is inherently differentiable, thus allowing end-to-end optimization. The audiogram is provided as an input to the DNN, thereby enabling listener-specific personalization without the need for retraining. Results show that the proposed approach not only allows adjusting the amounts of NR and HLC individually, but also improves objective metrics compared to optimizing a single training objective. It also outperforms a cascade of two DNNs that were separately trained for NR and HLC, and shows competitive HLC performance compared to a traditional hearing-aid prescription. To the best of our knowledge, this is the first study that uses an auditory model to train a single DNN for both NR and HLC across a wide range of listener profiles.

Motivation & Objective

• Develop a single DNN that jointly solves noise reduction (NR) and hearing loss compensation (HLC).
• Enable independent at-test adjustment of NR and HLC via mask exponentiation during inference.
• Personalize processing by incorporating the listener's audiogram without retraining.
• Use a differentiable auditory model to enable end-to-end optimization without emulator-based training.

Proposed method

• Define two time-frequency masks predicted by the DNN: one for NR and one for HLC.
• Train with distinct objectives for NR and HLC and balance them using an uncertainty-based weighting scheme.
• Combine the two masks at inference by exponentiating each mask with independent alphA_NR and alpha_HLC parameters.
• Model inputs include the audiogram to enable listener-specific personalization.
• Use a differentiable auditory model to provide a physiologically grounded, trainable target for both NR and HLC.

Experimental results

Research questions

• RQ1 Can a single DNN be trained end-to-end to perform both NR and HLC across a wide range of HI listeners?
• RQ2 Does multi-task training with separate objectives improve objective metrics over single-task or cascaded approaches?
• RQ3 Can NR and HLC be independently adjusted at inference without retraining?
• RQ4 Does incorporating an audiogram as input enable effective personalization without per-listener retraining?

Key findings

• The proposed approach enables independent adjustment of NR and HLC at inference via exponentiated masks.
• Multi-task training with uncertainty-based weighting improves objective metrics over single-objective optimization.
• The joint NR/HLC DNN outperforms a cascade of two separately trained DNNs for NR and HLC.
• The method shows competitive HLC performance compared to traditional hearing-aid prescriptions and works across a range of listener audiograms.
• To the authors’ knowledge, this is the first study using a differentiable auditory model to train a single DNN for both NR and HLC across diverse listener profiles.