[Paper Review] Global birdsong embeddings enable superior transfer learning for bioacoustic classification

Burooj Ghani, Tom Denton+2 more|arXiv (Cornell University)|Jul 12, 2023

Animal Vocal Communication and Behavior53 references4 citations

TL;DR

This study demonstrates that global birdsong embeddings from large-scale bird vocalization models enable superior few-shot transfer learning for bioacoustic classification across diverse taxa, including birds, bats, marine mammals, and amphibians. Trained on bird data, these embeddings outperform general audio embeddings in classifying rare species and subtle vocalizations with minimal labeled data.

ABSTRACT

Automated bioacoustic analysis aids understanding and protection of both marine and terrestrial animals and their habitats across extensive spatiotemporal scales, and typically involves analyzing vast collections of acoustic data. With the advent of deep learning models, classification of important signals from these datasets has markedly improved. These models power critical data analyses for research and decision-making in biodiversity monitoring, animal behaviour studies, and natural resource management. However, deep learning models are often data-hungry and require a significant amount of labeled training data to perform well. While sufficient training data is available for certain taxonomic groups (e.g., common bird species), many classes (such as rare and endangered species, many non-bird taxa, and call-type) lack enough data to train a robust model from scratch. This study investigates the utility of feature embeddings extracted from audio classification models to identify bioacoustic classes other than the ones these models were originally trained on. We evaluate models on diverse datasets, including different bird calls and dialect types, bat calls, marine mammals calls, and amphibians calls. The embeddings extracted from the models trained on bird vocalization data consistently allowed higher quality classification than the embeddings trained on general audio datasets. The results of this study indicate that high-quality feature embeddings from large-scale acoustic bird classifiers can be harnessed for few-shot transfer learning, enabling the learning of new classes from a limited quantity of training data. Our findings reveal the potential for efficient analyses of novel bioacoustic tasks, even in scenarios where available training data is limited to a few samples.

Motivation & Objective

• To evaluate the transferability of feature embeddings from audio classification models to novel bioacoustic tasks with limited training data.
• To compare the performance of embeddings trained on bird vocalizations versus general audio datasets for cross-taxonomic classification.
• To investigate the feasibility of using global birdsong embeddings for few-shot learning in low-data regimes common in conservation biology.
• To assess the effectiveness of embeddings in distinguishing subtle acoustic variations, such as bird song dialects and closely related species.
• To determine whether embeddings from specialized bird models generalize better than general-purpose audio models across diverse bioacoustic classes.

Proposed method

• Extracted feature embeddings from deep learning models pre-trained on large-scale bird vocalization datasets, such as BirdNET.
• Applied these embeddings as input features for linear classifiers in a few-shot learning setup with limited labeled examples.
• Evaluated performance across diverse datasets: bird calls, dialects, bat echolocation, marine mammal vocalizations, and amphibian calls.
• Used standard metrics like accuracy, F1-score, and confusion matrices to compare model performance across different embedding sources.
• Conducted ablation studies to assess the impact of embedding dimensionality on classification performance.
• Addressed class imbalance and co-occurrence confusions by analyzing misclassification patterns in overlapping species (e.g., bearded seal and bowhead whale).

Figure 1 : Workflow illustrating the process of downstream classification for various bioacoustic tasks. Spectrograms are processed by pre-trained models, producing embeddings. These embeddings are then passed to a classifier head—a fully-connected feed forward neural network—which is trained for fi

Experimental results

Research questions

• RQ1 Can feature embeddings from large-scale bird vocalization models generalize effectively to unseen bioacoustic classes with few labeled examples?
• RQ2 Do embeddings trained on bird-specific data outperform those from general audio datasets in cross-taxonomic bioacoustic classification?
• RQ3 How does embedding size affect classification performance in low-data regimes?
• RQ4 What are the main sources of misclassification in complex acoustic environments with co-occurring species?
• RQ5 Can embeddings capture subtle acoustic differences, such as bird song dialects, that are difficult to distinguish with traditional methods?

Key findings

• Embeddings from bird-specific models consistently outperformed general audio embeddings across all tested datasets, including bat, marine mammal, and amphibian vocalizations.
• The study achieved superior classification accuracy in few-shot regimes, demonstrating that global birdsong embeddings enable effective transfer learning with as few as a few labeled examples.
• Confusion between bearded seals and bowhead whales was high (18.6%) due to overlapping ranges and vocal similarities, indicating a need for better training data curation.
• The YD dataset showed high variance in low-data regimes, likely due to the subtle acoustic distinction between dialects based on note order rather than amplitude or timbre.
• Larger embedding sizes improved separability of acoustic classes but increased model size and reduced inference speed, leading to larger BirdNET versions with enhanced performance.
• The results support the hypothesis that embeddings from models trained on closely related data (e.g., bird vocalizations) generalize better than those from unrelated domains, even across taxonomic boundaries.

Figure 2 : Results of Few-Shot Transfer Learning Tasks. ROC-AUC scores are plotted with log-odds scaling. A point is plotted for each experiment, and the curve connects the average quality for experiments at each number of training examples per class.