[论文解读] End-to-End Multi-Task Learning for Adjustable Joint Noise Reduction and Hearing Loss Compensation

Philippe Gonzalez, Vera Margrethe Frederiksen+2 more|arXiv (Cornell University)|Mar 20, 2026

Hearing Loss and Rehabilitation被引用 0

一句话总结

作者提出一个端到端的多任务深度神经网络，在推断阶段可独立调整降噪与听力损失补偿的程度，并通过聆听者的听力图输入进行个性化，同时使用可微分听觉模型进行训练以实现端到端优化。

ABSTRACT

A multi-task learning framework is proposed for optimizing a single deep neural network (DNN) for joint noise reduction (NR) and hearing loss compensation (HLC). A distinct training objective is defined for each task, and the DNN predicts two time-frequency masks. During inference, the amounts of NR and HLC can be adjusted independently by exponentiating each mask before combining them. In contrast to recent approaches that rely on training an auditory-model emulator to define a differentiable training objective, we propose an auditory model that is inherently differentiable, thus allowing end-to-end optimization. The audiogram is provided as an input to the DNN, thereby enabling listener-specific personalization without the need for retraining. Results show that the proposed approach not only allows adjusting the amounts of NR and HLC individually, but also improves objective metrics compared to optimizing a single training objective. It also outperforms a cascade of two DNNs that were separately trained for NR and HLC, and shows competitive HLC performance compared to a traditional hearing-aid prescription. To the best of our knowledge, this is the first study that uses an auditory model to train a single DNN for both NR and HLC across a wide range of listener profiles.

研究动机与目标

• 开发一个单一的DNN，联合解决降噪（NR）与听力损失补偿（HLC）。
• 通过推断阶段的掩码指数化，实现NR和HLC在测试时的独立调节。
• 在不重新训练的情况下，通过纳入听力图实现处理个性化。
• 使用可微分的听觉模型，使端到端优化成为可能，而无需基于仿真器的训练。

提出的方法

• 由DNN预测的两个时-频掩码：一个用于NR，一个用于HLC。
• 用NR和HLC的不同目标进行训练，并通过基于不确定性的加权方案对它们进行平衡。
• 在推断时通过以独立的alphA_NR和alpha_HLC参数对每个掩码进行指数化来组合两个掩码。
• 模型输入包括听力图以实现聆听者特定的个性化。
• 使用可微分的听觉模型，为NR和HLC提供一个生理学上有依据且可训练的目标。

实验结果

研究问题

• RQ1 是否可以训练一个单一的DNN端到端地在广泛的听障者人群中同时执行NR和HLC？
• RQ2 带有独立目标的多任务训练是否在客观指标上优于单任务或级联方法？
• RQ3 是否可以在推断阶段独立调整NR和HLC而无需重新训练？
• RQ4 将听力图作为输入是否可以在不对每个听者重新训练的情况下实现有效个性化？

主要发现

• 所提出的方法通过对掩码进行指数化，在推断阶段实现NR和HLC的独立调节。
• 带有基于不确定性的加权的多任务训练在客观指标上优于单一目标优化。
• 联合NR/HLC的DNN在性能上优于对NR和HLC分别训练的两步级联DNN。
• 该方法在与传统助听器处方相比的HLC性能方面具有竞争力，且适用于不同听力图的听众。
• 据作者所知，这是首个使用可微分听觉模型来训练单一DNN，在覆盖多样化听众配置的情况下同时实现NR和HLC的研究。