[Paper Review] Learning to match transient sound events using attentional similarity for few-shot sound recognition

Szu-Yu Chou, Kai-Hsiang Cheng+2 more|arXiv (Cornell University)|Dec 4, 2018

Music and Audio Processing22 references3 citations

TL;DR

This paper proposes an attentional similarity module that enhances few-shot sound recognition by focusing on transient audio events through learnable attention mechanisms. Integrated into metric-based learning frameworks, it improves matching accuracy for short, distinct sounds, achieving relative gains of 4.1% to 7.7% on ESC-50 and 2.1% to 6.5% on noisy ESC-50 across five baseline methods.

ABSTRACT

In this paper, we introduce a novel attentional similarity module for the problem of few-shot sound recognition. Given a few examples of an unseen sound event, a classifier must be quickly adapted to recognize the new sound event without much fine-tuning. The proposed attentional similarity module can be plugged into any metric-based learning method for few-shot learning, allowing the resulting model to especially match related short sound events. Extensive experiments on two datasets shows that the proposed module consistently improves the performance of five different metric-based learning methods for few-shot sound recognition. The relative improvement ranges from +4.1% to +7.7% for 5-shot 5-way accuracy for the ESC-50 dataset, and from +2.1% to +6.5% for noiseESC-50. Qualitative results demonstrate that our method contributes in particular to the recognition of transient sound events.

Motivation & Objective

• To address the challenge of recognizing rare or transient sound events with only a few training examples.
• To improve few-shot sound recognition performance by enhancing similarity matching in metric-based learning frameworks.
• To develop a plug-in module that guides models to focus on relevant audio segments without requiring instance-level annotations.
• To evaluate the method on both clean and noisy audio benchmarks to assess robustness in real-world conditions.

Proposed method

• The attentional similarity module replaces standard similarity functions with a learnable attention mechanism that highlights relevant temporal segments in audio clips.
• It computes attention weights over the feature maps of support and query examples, emphasizing short, transient events over background noise.
• The module is differentiable and can be integrated into any metric-based few-shot learning model, such as Prototypical Networks or Matching Networks.
• The attention mechanism is trained using clip-level labels only, avoiding the need for weakly supervised instance-level annotations.
• The backbone network uses a simple CNN architecture with 3×3 convolutions, batch normalization, ReLU, and 4×4 max-pooling layers on log-mel spectrograms.
• Training employs cross-entropy loss with SGD, weight decay, and learning rate decay, using z-score normalized mel-spectrogram features from 16 kHz audio.

Experimental results

Research questions

• RQ1 Can an attentional similarity module improve few-shot sound recognition performance, especially for transient sound events?
• RQ2 How does the attentional similarity module enhance matching accuracy compared to standard similarity functions in metric-based learning?
• RQ3 Does the module generalize across different metric-based learning frameworks and datasets, including noisy audio?
• RQ4 To what extent does the module reduce interference from background noise during few-shot classification?
• RQ5 Can the module be trained effectively using only clip-level annotations without requiring instance-level labels?

Key findings

• The attentional similarity module improves 5-way 5-shot accuracy by 4.1% to 7.7% on the clean ESC-50 dataset across five metric-based learning methods.
• On the noisy ESC-50 dataset, the module achieves relative gains of 2.1% to 6.5% in 5-way 5-shot learning, demonstrating robustness to environmental noise.
• The prototypical network with the attentional similarity module achieves the highest performance, reaching 74.2% accuracy on 5-way 5-shot ESC-50 and 65.7% on noisy ESC-50.
• Qualitative analysis shows that the model with attention correctly matches transient events, while the baseline model often selects clips with similar silence duration instead.
• The module effectively suppresses background noise and improves focus on short-duration events, as evidenced by attention maps highlighting relevant audio segments.
• The method is universally applicable and consistently improves performance across diverse metric-based models, indicating strong generalization capability.