[Paper Review] Guided Learning Convolution System for DCASE 2019 Task 4

Liwei Lin, Xiangdong Wang+2 more|arXiv (Cornell University)|Sep 11, 2019

Music and Audio Processing12 references4 citations

TL;DR

This paper presents a guided learning convolutional neural network system for weakly-supervised and semi-supervised sound event detection in domestic environments using weakly labeled and unlabeled data. By integrating embedding-level attention pooling, disentangled feature learning, and adaptive median filtering, the model achieves a 42.7% event-based F-measure on the DCASE2019 test set, outperforming all other entries in the challenge.

ABSTRACT

In this paper, we describe in detail the system we submitted to DCASE2019 task 4: sound event detection (SED) in domestic environments. We employ a convolutional neural network (CNN) with an embedding-level attention pooling module to solve it. By considering the interference caused by the co-occurrence of multiple events in the unbalanced dataset, we utilize the disentangled feature to raise the performance of the model. To take advantage of the unlabeled data, we adopt Guided Learning for semi-supervised learning. A group of median filters with adaptive window sizes is utilized in the post-processing of output probabilities of the model. We also analyze the effect of the synthetic data on the performance of the model and finally achieve an event-based F-measure of 45.43% on the validation set and an event-based F-measure of 42.7% on the test set. The system we submitted to the challenge achieves the best performance compared to those of other participates.

Motivation & Objective

• To address the challenge of large-scale sound event detection in domestic environments using weakly-labeled and unlabeled audio data.
• To improve model performance on unbalanced datasets with co-occurring sound events through disentangled feature learning.
• To effectively combine weakly-supervised and semi-supervised learning using the Guided Learning framework.
• To develop an adaptive post-processing method that improves boundary detection accuracy based on event duration characteristics.
• To evaluate the impact of a strongly annotated synthetic training set on model generalization and performance.

Proposed method

• A CNN model with embedding-level attention pooling is used to generate clip-level and frame-level predictions from high-level features, enabling weakly-supervised learning.
• The attention mechanism computes frame-wise weights via a learnable vector and bias, allowing the model to focus on key audio frames for each event class.
• Disentangled feature learning is applied to reduce interference from co-occurring events, improving robustness in multi-event scenarios.
• Guided Learning is employed for semi-supervised training, leveraging unlabeled data to improve generalization.
• Adaptive median filtering with varying window sizes is applied to output probabilities to refine event boundary detection based on event duration.
• The synthetic training set is treated as weakly-labeled data and evaluated through ablation studies to assess its impact on performance.

Experimental results

Research questions

• RQ1 How can a deep learning model effectively learn from weakly-labeled audio data without timestamp annotations?
• RQ2 How can model performance be improved on unbalanced datasets with overlapping sound events?
• RQ3 Can guided learning effectively combine weakly-labeled and unlabeled data to boost performance in semi-supervised SED?
• RQ4 Does adaptive post-processing with median filtering improve boundary detection accuracy across events with varying durations?
• RQ5 Does the inclusion of a synthetic training set with strong annotations enhance performance in weakly-supervised or semi-supervised settings?

Key findings

• The proposed system achieved an event-based F-measure of 42.7% on the DCASE2019 test set, securing first place in the challenge.
• The ensemble of top-6 models (Ensemble Top1-6) improved performance by 21.73 percentage points over the baseline.
• Models using Guided Learning with both weakly-labeled and unlabeled data outperformed those using only weakly-labeled data by 20.67 percentage points.
• The synthetic training set did not help in weakly-supervised learning alone but significantly improved performance when combined with semi-supervised learning, boosting F1 by 5–8 points.
• Removing disentangled feature learning or fixed-window median filtering degraded performance, confirming their effectiveness.
• The model achieved the best performance on the YouTube dataset but underperformed on Vimeo, suggesting data augmentation like pitch shifting may be beneficial.