[Paper Review] Learning to detect an animal sound from five examples

Inês Nolasco, Shubhr Singh+15 more|arXiv (Cornell University)|May 22, 2023

Animal Vocal Communication and BehaviorBiochemistry, Genetics and Molecular Biology60 references3 citations

TL;DR

This paper introduces few-shot bioacoustic sound event detection (FSED), enabling models to detect animal vocalizations from just five annotated examples. By adapting prototypical networks with audio-specific preprocessing and query-time adaptation, the method achieves strong performance on diverse, low-resource bioacoustic tasks, outperforming traditional signal processing and demonstrating the feasibility of generalizable, few-shot models for wildlife sound analysis.

ABSTRACT

Automatic detection and classification of animal sounds has many applications in biodiversity monitoring and animal behaviour. In the past twenty years, the volume of digitised wildlife sound available has massively increased, and automatic classification through deep learning now shows strong results. However, bioacoustics is not a single task but a vast range of small-scale tasks (such as individual ID, call type, emotional indication) with wide variety in data characteristics, and most bioacoustic tasks do not come with strongly-labelled training data. The standard paradigm of supervised learning, focussed on a single large-scale dataset and/or a generic pre-trained algorithm, is insufficient. In this work we recast bioacoustic sound event detection within the AI framework of few-shot learning. We adapt this framework to sound event detection, such that a system can be given the annotated start/end times of as few as 5 events, and can then detect events in long-duration audio -- even when the sound category was not known at the time of algorithm training. We introduce a collection of open datasets designed to strongly test a system's ability to perform few-shot sound event detections, and we present the results of a public contest to address the task. We show that prototypical networks are a strong-performing method, when enhanced with adaptations for general characteristics of animal sounds. We demonstrate that widely-varying sound event durations are an important factor in performance, as well as non-stationarity, i.e. gradual changes in conditions throughout the duration of a recording. For fine-grained bioacoustic recognition tasks without massive annotated training data, our results demonstrate that few-shot sound event detection is a powerful new method, strongly outperforming traditional signal-processing detection methods in the fully automated scenario.

Motivation & Objective

• Address the challenge of low-resource bioacoustic sound event detection, where labeled data is scarce and tasks vary widely in sound characteristics.
• Develop a few-shot learning framework tailored for animal vocalizations, capable of generalizing across diverse species and call types with minimal supervision.
• Evaluate the performance of few-shot models on real-world, long-duration audio recordings with non-stationary conditions and variable event durations.
• Demonstrate that prototype-based meta-learning with query-time adaptation can achieve state-of-the-art results in few-shot bioacoustic detection.
• Promote the development of reusable, general-purpose audio embeddings for bioacoustic analysis through a public challenge and open datasets.

Proposed method

• Adapt prototypical networks for few-shot sound event detection (FSED), using support sets of five annotated animal vocalizations and background audio to define new tasks.
• Apply per-channel energy normalization (PCEN) to enhance robustness to non-stationary audio conditions and improve feature representation.
• Implement query-time adaptation (transductive inference) to fine-tune model predictions on new audio clips, improving detection accuracy without retraining.
• Use duration filtering and postprocessing to handle variable event lengths, which significantly impact performance in real-world recordings.
• Introduce a public benchmark with diverse, open datasets covering multiple species and call types to rigorously test few-shot generalization.
• Evaluate both prototypical and non-prototypical approaches, including fine-tuning and query-time adaptive weighting (e.g., DFSL), to compare generalization strategies.

Figure 1: (a) Few-shot sound event detection: the first 5 sound events are given as examples—in standard supervised learning they would be considered the training set—and the remainder must then be detected. (b) Few-shot sound event detection as a meta-learning problem. Each of our datasets represen

Experimental results

Research questions

• RQ1 Can few-shot learning be effectively applied to bioacoustic sound event detection with only five labeled examples per class?
• RQ2 How do non-stationarity and variable event durations in long audio recordings affect few-shot detection performance?
• RQ3 Does query-time adaptation significantly improve detection accuracy in few-shot bioacoustic tasks?
• RQ4 Can a single, fixed embedding space generalize across diverse bioacoustic tasks without test-time adaptation?
• RQ5 How do prototype-based meta-learning and alternative fine-tuning methods compare in performance on real-world FSED benchmarks?

Key findings

• Prototype-based meta-learning with proper negative example selection and duration filtering achieves strong performance on few-shot bioacoustic sound event detection.
• Query-time adaptation significantly improves detection accuracy, especially in non-stationary recordings, but adds computational cost and complexity.
• Even without query-time adaptation, the best prototypical network models produce powerful, reusable embeddings that generalize across diverse animal vocalizations.
• The method outperforms traditional signal-processing-based detection in fully automated scenarios, particularly on low-resource and variable-duration tasks.
• Non-prototypical approaches such as fine-tuning and DFSL (query-time adaptive weighting) also achieve strong results, indicating that meta-learning is not strictly necessary.
• The introduction of ensemble restrictions in the 2023 challenge pushes models toward generalization, favoring single, robust models over ensemble-based solutions.

Figure 2: Sample spectrograms for each dataset. POS (positive, i.e. target) vocalizations are indicated with a white rectangle.