[论文解读] Global birdsong embeddings enable superior transfer learning for bioacoustic classification

Burooj Ghani, Tom Denton+2 more|arXiv (Cornell University)|Jul 12, 2023

Animal Vocal Communication and Behavior参考文献 53被引用 4

一句话总结

本研究证明，基于大规模鸟类鸣叫模型的全局鸟类鸣叫嵌入在跨多样物种类（包括鸟类、蝙蝠、海洋哺乳动物和两栖动物）的生物声学分类任务中，能够实现更优的 few-shot 迁移学习。这些在鸟类数据上训练的嵌入在仅使用少量标注数据的情况下，对稀有物种和细微鸣叫的分类表现优于通用音频嵌入。

ABSTRACT

Automated bioacoustic analysis aids understanding and protection of both marine and terrestrial animals and their habitats across extensive spatiotemporal scales, and typically involves analyzing vast collections of acoustic data. With the advent of deep learning models, classification of important signals from these datasets has markedly improved. These models power critical data analyses for research and decision-making in biodiversity monitoring, animal behaviour studies, and natural resource management. However, deep learning models are often data-hungry and require a significant amount of labeled training data to perform well. While sufficient training data is available for certain taxonomic groups (e.g., common bird species), many classes (such as rare and endangered species, many non-bird taxa, and call-type) lack enough data to train a robust model from scratch. This study investigates the utility of feature embeddings extracted from audio classification models to identify bioacoustic classes other than the ones these models were originally trained on. We evaluate models on diverse datasets, including different bird calls and dialect types, bat calls, marine mammals calls, and amphibians calls. The embeddings extracted from the models trained on bird vocalization data consistently allowed higher quality classification than the embeddings trained on general audio datasets. The results of this study indicate that high-quality feature embeddings from large-scale acoustic bird classifiers can be harnessed for few-shot transfer learning, enabling the learning of new classes from a limited quantity of training data. Our findings reveal the potential for efficient analyses of novel bioacoustic tasks, even in scenarios where available training data is limited to a few samples.

研究动机与目标

• 评估音频分类模型的特征嵌入在有限训练数据下向新型生物声学任务迁移的可行性。
• 比较在鸟类鸣叫数据与通用音频数据集上训练的嵌入在跨物种类分类中的性能表现。
• 探究在保护生物学中常见的低数据环境下，使用全局鸟类鸣叫嵌入实现 few-shot 学习的可行性。
• 评估嵌入在区分细微声学差异（如鸟类鸣叫方言和近缘物种）方面的有效性。
• 确定专门针对鸟类的模型生成的嵌入是否在跨多样化生物声学类别中比通用音频模型的嵌入泛化能力更强。

提出的方法

• 从在大规模鸟类鸣叫数据集（如 BirdNET）上预训练的深度学习模型中提取特征嵌入。
• 在 few-shot 学习设置中，将这些嵌入作为线性分类器的输入特征，仅使用少量标注样本。
• 在多种数据集上评估性能：鸟类鸣叫、方言、蝙蝠回声定位、海洋哺乳动物发声以及两栖动物叫声。
• 使用准确率、F1 分数和混淆矩阵等标准指标，比较不同嵌入来源的模型性能。
• 通过消融研究评估嵌入维度对分类性能的影响。
• 通过分析重叠物种类（如髯海豹与 bowhead 鲸）的误分类模式，解决类别不平衡和共现混淆问题。

Figure 1 : Workflow illustrating the process of downstream classification for various bioacoustic tasks. Spectrograms are processed by pre-trained models, producing embeddings. These embeddings are then passed to a classifier head—a fully-connected feed forward neural network—which is trained for fi

实验结果

研究问题

• RQ1 来自大规模鸟类鸣叫模型的特征嵌入是否能在仅使用少量标注样本的情况下，有效泛化到未见的生物声学类别？
• RQ2 在跨物种类生物声学分类中，基于鸟类特定数据训练的嵌入是否优于通用音频数据集训练的嵌入？
• RQ3 在低数据环境下，嵌入大小如何影响分类性能？
• RQ4 在存在共现物种的复杂声学环境中，误分类的主要来源是什么？
• RQ5 嵌入是否能够捕捉到传统方法难以区分的细微声学差异（如鸟类鸣叫方言）？

主要发现

• 在所有测试数据集（包括蝙蝠、海洋哺乳动物和两栖动物发声）中，基于鸟类特定模型的嵌入始终优于通用音频嵌入。
• 研究在 few-shot 环境中实现了更高的分类准确率，表明全局鸟类鸣叫嵌入可在仅数个标注样本下实现有效的迁移学习。
• 由于分布重叠和发声相似性，髯海豹与 bowhead 鲸之间的误分类率较高（18.6%），表明需要更优的训练数据整理。
• YD 数据集在低数据环境下表现出较高的方差，可能是因为方言之间的细微差异基于音符顺序而非振幅或音色。
• 更大的嵌入尺寸提升了声学类别的可分性，但同时也增加了模型大小并降低了推理速度，因此推出了性能更强的更大版本 BirdNET。
• 结果支持如下假设：在相关数据（如鸟类鸣叫）上训练的模型生成的嵌入，即使在跨物种类边界时，其泛化能力也优于来自无关领域的嵌入。

Figure 2 : Results of Few-Shot Transfer Learning Tasks. ROC-AUC scores are plotted with log-odds scaling. A point is plotted for each experiment, and the curve connects the average quality for experiments at each number of training examples per class.