[论文解读] Learning to detect an animal sound from five examples

Inês Nolasco, Shubhr Singh+15 more|arXiv (Cornell University)|May 22, 2023

Animal Vocal Communication and BehaviorBiochemistry, Genetics and Molecular Biology参考文献 60被引用 3

一句话总结

本文提出少样本生物声学事件检测（FSED），使模型仅需五个标注样本即可检测动物发声。通过结合音频特定预处理与查询时适应的原型网络方法，该方法在多样化的低资源生物声学任务中表现优异，优于传统信号处理方法，证明了通用少样本模型在野生动物声学分析中的可行性。

ABSTRACT

Automatic detection and classification of animal sounds has many applications in biodiversity monitoring and animal behaviour. In the past twenty years, the volume of digitised wildlife sound available has massively increased, and automatic classification through deep learning now shows strong results. However, bioacoustics is not a single task but a vast range of small-scale tasks (such as individual ID, call type, emotional indication) with wide variety in data characteristics, and most bioacoustic tasks do not come with strongly-labelled training data. The standard paradigm of supervised learning, focussed on a single large-scale dataset and/or a generic pre-trained algorithm, is insufficient. In this work we recast bioacoustic sound event detection within the AI framework of few-shot learning. We adapt this framework to sound event detection, such that a system can be given the annotated start/end times of as few as 5 events, and can then detect events in long-duration audio -- even when the sound category was not known at the time of algorithm training. We introduce a collection of open datasets designed to strongly test a system's ability to perform few-shot sound event detections, and we present the results of a public contest to address the task. We show that prototypical networks are a strong-performing method, when enhanced with adaptations for general characteristics of animal sounds. We demonstrate that widely-varying sound event durations are an important factor in performance, as well as non-stationarity, i.e. gradual changes in conditions throughout the duration of a recording. For fine-grained bioacoustic recognition tasks without massive annotated training data, our results demonstrate that few-shot sound event detection is a powerful new method, strongly outperforming traditional signal-processing detection methods in the fully automated scenario.

研究动机与目标

• 解决低资源生物声学事件检测的挑战，即标注数据稀缺且各类任务在声音特征上差异显著。
• 开发针对动物发声的少样本学习框架，能够在极少监督下跨多种物种和叫声类型实现泛化。
• 在具有非平稳条件和可变事件时长的真实长时音频记录中评估少样本模型的性能。
• 证明基于原型的元学习结合查询时适应可在少样本生物声学检测中实现最先进性能。
• 通过公开挑战和开放数据集，推动可复用、通用音频嵌入在生物声学分析中的发展。

提出的方法

• 将原型网络适配用于少样本声音事件检测（FSED），使用包含五个标注动物发声及背景音频的支持集来定义新任务。
• 应用每通道能量归一化（PCEN）以增强对非平稳音频条件的鲁棒性，并改善特征表示。
• 实施查询时适应（归纳推理）以在新音频片段上微调模型预测，提升检测准确率，而无需重新训练。
• 采用时长过滤与后处理方法以应对可变事件长度，该因素在真实录音中显著影响性能。
• 引入公开基准，包含覆盖多种物种和叫声类型的多样化开放数据集，以严格测试少样本泛化能力。
• 评估原型与非原型方法，包括微调和查询时自适应加权（如DFSL），以比较不同泛化策略。

Figure 1: (a) Few-shot sound event detection: the first 5 sound events are given as examples—in standard supervised learning they would be considered the training set—and the remainder must then be detected. (b) Few-shot sound event detection as a meta-learning problem. Each of our datasets represen

实验结果

研究问题

• RQ1 仅对每类提供五个标注样本，少样本学习能否有效应用于生物声学事件检测？
• RQ2 长音频记录中的非平稳性与可变事件时长如何影响少样本检测性能？
• RQ3 查询时适应是否显著提升少样本生物声学任务中的检测准确率？
• RQ4 是否存在一个单一、固定的嵌入空间，可在无需测试时适应的情况下泛化至多样化生物声学任务？
• RQ5 在真实世界FSED基准上，基于原型的元学习与替代微调方法在性能上如何比较？

主要发现

• 通过合理选择负样本并结合时长过滤，基于原型的元学习在少样本生物声学事件检测中表现强劲。
• 查询时适应显著提升检测准确率，尤其在非平稳录音中，但会增加计算成本与复杂性。
• 即使不使用查询时适应，最佳原型网络模型仍能生成强大且可复用的嵌入，泛化能力覆盖多种动物发声。
• 该方法在全自动场景中优于传统基于信号处理的检测方法，尤其在低资源与可变时长效务中表现更优。
• 非原型方法如微调与DFSL（查询时自适应加权）也取得优异结果，表明元学习并非严格必需。
• 2023年挑战赛中引入的集成限制促使模型更注重泛化，更青睐单一稳健模型而非基于集成的解决方案。

Figure 2: Sample spectrograms for each dataset. POS (positive, i.e. target) vocalizations are indicated with a white rectangle.