[Paper Review] Leveraging tropical reef, bird and unrelated sounds for superior transfer learning in marine bioacoustics

Ben Williams, Bart van Merriënboer+13 more|arXiv (Cornell University)|Apr 25, 2024

Underwater Acoustics Research5 citations

TL;DR

The paper shows that cross-domain pretraining using bird, reef, and unrelated sounds yields superior transfer learning for marine bioacoustics, with SurfPerch as a strong pretrained model for low-annotation PAM tasks.

ABSTRACT

Machine learning has the potential to revolutionize passive acoustic monitoring (PAM) for ecological assessments. However, high annotation and compute costs limit the field's efficacy. Generalizable pretrained networks can overcome these costs, but high-quality pretraining requires vast annotated libraries, limiting its current applicability primarily to bird taxa. Here, we identify the optimum pretraining strategy for a data-deficient domain using coral reef bioacoustics. We assemble ReefSet, a large annotated library of reef sounds, though modest compared to bird libraries at 2% of the sample count. Through testing few-shot transfer learning performance, we observe that pretraining on bird audio provides notably superior generalizability compared to pretraining on ReefSet or unrelated audio alone. However, our key findings show that cross-domain mixing which leverages bird, reef and unrelated audio during pretraining maximizes reef generalizability. SurfPerch, our pretrained network, provides a strong foundation for automated analysis of marine PAM data with minimal annotation and compute costs.

Motivation & Objective

• Motivate reducing annotation and compute costs in passive acoustic monitoring (PAM) for ecological assessments.
• Identify optimal pretraining strategy for data-deficient coral reef bioacoustics.
• Assess generalizability gains from cross-domain pretraining across taxa and sound types.
• Provide a pretrained network (SurfPerch) as a foundation for marine PAM analysis.

Proposed method

• Assemble ReefSet, a large annotated library of reef sounds (2% of bird library size by sample count).
• Evaluate few-shot transfer learning performance to compare pretraining on bird, ReefSet, and unrelated audio.
• Test cross-domain mixing that combines bird, reef, and unrelated audio during pretraining to maximize reef generalizability.
• Develop and release SurfPerch, a pretrained network for automated marine PAM analysis.

Experimental results

Research questions

• RQ1 What pretraining source yields the best generalization for reef-related marine bioacoustic tasks?
• RQ2 Does mixing across domains (bird, reef, unrelated sounds) improve transfer learning for reef species detection?
• RQ3 Can a pretrained network trained on cross-domain data reduce annotation and compute costs in marine PAM?

Key findings

• Pretraining on bird audio provides notably superior generalizability for reef data compared to ReefSet or unrelated audio alone.
• Cross-domain mixing during pretraining maximizes reef generalizability.
• SurfPerch serves as a strong foundation for automated marine PAM analysis with minimal annotation and compute costs.