[Paper Review] L3DAS22 Challenge: Learning 3D Audio Sources in a Real Office Environment

Eric Guizzo, C. Marinoni+7 more|arXiv (Cornell University)|Feb 21, 2022

Speech and Audio Processing4 citations

TL;DR

This paper presents the L3DAS22 Challenge, a machine learning grand challenge focused on 3D audio source separation and localization in real office environments using Ambisonics recordings. It introduces a new 98-hour MSMP B-format dataset, improves baseline models using top-performing architectures (U-Net for speech enhancement, modified SELDnet for sound localization), and reports state-of-the-art results: 0.984 F1-score in 3D speech enhancement and 0.699 in 3D sound event localization and detection.

ABSTRACT

The L3DAS22 Challenge is aimed at encouraging the development of machine learning strategies for 3D speech enhancement and 3D sound localization and detection in office-like environments. This challenge improves and extends the tasks of the L3DAS21 edition. We generated a new dataset, which maintains the same general characteristics of L3DAS21 datasets, but with an extended number of data points and adding constrains that improve the baseline model's efficiency and overcome the major difficulties encountered by the participants of the previous challenge. We updated the baseline model of Task 1, using the architecture that ranked first in the previous challenge edition. We wrote a new supporting API, improving its clarity and ease-of-use. In the end, we present and discuss the results submitted by all participants. L3DAS22 Challenge website: www.l3das.com/icassp2022.

Motivation & Objective

• To advance machine learning for 3D audio processing in real-world office environments with complex acoustics.
• To address limitations in the L3DAS21 challenge by expanding dataset size, improving data synthesis, and reducing computational demands.
• To provide updated, high-performing baseline models for 3D speech enhancement (SE) and 3D sound event localization and detection (SELD).
• To facilitate reproducible research through a redesigned, clearer, and faster supporting API for data preprocessing and model training.
• To evaluate and benchmark novel deep learning approaches across two core tasks: 3D SE and 3D SELD using real-world Ambisonics recordings.

Proposed method

• Collected 98 hours of multi-source, multi-perspective (MSMP) B-format Ambisonics recordings using two first-order Soundfield microphones in a real office room.
• Synthesized a dataset with 252 speaker positions (168 grid + 84 random) using 20-second exponential sine sweeps to estimate 3D impulse responses.
• Developed a U-Net-based model for 3D speech enhancement that estimates time-frequency masks to enhance noisy B-format signals via beamforming.
• Modified SELDnet architecture for 3D SELD: used PyTorch, increased network capacity, removed phase information, applied max-pooling over time and frequency, and enabled detection of up to three simultaneous sources of the same class.
• Implemented a new, improved API with better clarity, performance, and bug fixes to streamline data preprocessing and baseline training/evaluation.
• Enabled participants to use both 1-mic and 2-mic configurations, and to augment data or use pretrained models, with no restrictions on methodological approaches.

Experimental results

Research questions

• RQ1 How can a large-scale, real-world 3D audio dataset be effectively synthesized to support robust 3D speech enhancement and sound localization tasks?
• RQ2 What deep learning architectures yield the best performance in 3D speech enhancement and 3D sound event localization and detection using Ambisonics signals?
• RQ3 How can baseline models be improved to reduce computational cost while maintaining or improving performance over previous challenges?
• RQ4 What are the key challenges in 3D audio processing in reverberant environments, and how can they be mitigated through data and model design?
• RQ5 To what extent can participants improve upon the baseline models using diverse architectures, data augmentation, or transfer learning in the L3DAS22 Challenge?

Key findings

• The L3DAS22 Challenge dataset comprises 98 hours of high-fidelity, multi-source, multi-perspective B-format recordings from a real office environment with controlled reverberation.
• The winning team for 3D speech enhancement (ESP-SE) achieved a T1 metric score of 0.984, with a word error rate of 0.019 and STOI of 0.987, significantly outperforming the baseline of 0.83.
• The winning team for 3D SELD (Lab9_DSP411) achieved a T2 metric score of 0.699, with precision of 0.706 and recall of 0.691, surpassing the baseline F-score of 0.34.
• Several teams improved upon the baseline models in both tasks, demonstrating the effectiveness of advanced architectures and data augmentation strategies.
• The new API significantly improved the efficiency and usability of data preprocessing and baseline training, supporting faster iteration and reproducibility.
• The challenge received 46 registrations and 24 submissions, with 17 teams submitting results for 3D SE and 7 for 3D SELD, indicating strong community engagement in 3D audio machine learning.