[Paper Review] Towards Open Respiratory Acoustic Foundation Models: Pretraining and Benchmarking

Yuwei Zhang, Xia Tong+7 more|arXiv (Cornell University)|Jun 23, 2024

Acoustic Wave Phenomena Research4 citations

TL;DR

This paper introduces OPERA, an open-source framework for pretraining and benchmarking respiratory acoustic foundation models using a large-scale, multi-source dataset of 136K respiratory audio samples (440 hours). The system pretrains three foundation models via contrastive and generative self-supervision, and benchmarks them across 19 downstream tasks, demonstrating superior performance over general audio models on 16/19 tasks, highlighting the value of domain-specific pretraining for respiratory health applications.

ABSTRACT

Respiratory audio, such as coughing and breathing sounds, has predictive power for a wide range of healthcare applications, yet is currently under-explored. The main problem for those applications arises from the difficulty in collecting large labeled task-specific data for model development. Generalizable respiratory acoustic foundation models pretrained with unlabeled data would offer appealing advantages and possibly unlock this impasse. However, given the safety-critical nature of healthcare applications, it is pivotal to also ensure openness and replicability for any proposed foundation model solution. To this end, we introduce OPERA, an OPEn Respiratory Acoustic foundation model pretraining and benchmarking system, as the first approach answering this need. We curate large-scale respiratory audio datasets (~136K samples, over 400 hours), pretrain three pioneering foundation models, and build a benchmark consisting of 19 downstream respiratory health tasks for evaluation. Our pretrained models demonstrate superior performance (against existing acoustic models pretrained with general audio on 16 out of 19 tasks) and generalizability (to unseen datasets and new respiratory audio modalities). This highlights the great promise of respiratory acoustic foundation models and encourages more studies using OPERA as an open resource to accelerate research on respiratory audio for health. The system is accessible from https://github.com/evelyn0414/OPERA.

Motivation & Objective

• Address the lack of open, large-scale respiratory audio datasets and foundation models for healthcare applications.
• Overcome the challenge of limited labeled data in respiratory audio modeling by enabling self-supervised pretraining on unlabeled data.
• Establish a comprehensive, reproducible benchmark for evaluating respiratory acoustic foundation models across diverse health tasks.
• Ensure openness and replicability in respiratory AI research by releasing curated data, models, and evaluation protocols.
• Demonstrate the superiority of domain-specific foundation models over general-purpose audio models in respiratory health applications.

Proposed method

• Curated a large-scale, multi-source respiratory audio dataset (136K samples, 440 hours) from five public sources, including coughing, breathing, and lung sounds.
• Pretrained three foundation models using self-supervised learning: a contrastive learning-based transformer (OPERA-CT), a contrastive CNN (OPERA-CE), and a generative transformer (OPERA-GT) with masked spectrogram reconstruction.
• Employed standard contrastive learning objectives (InfoNCE loss) and a generative reconstruction objective (MSE loss on spectrograms) to train the models.
• Formulated 19 downstream tasks across two categories: 12 for health condition inference (e.g., COPD, asthma, flu) and 7 for lung function estimation (e.g., FVC, FEV1).
• Evaluated models via linear probing and fine-tuning on 10 labeled datasets, including 6 not used in pretraining, ensuring fair and generalizable benchmarking.
• Used standard metrics: AUROC for classification, MAE and MAPE for regression, with t-SNE visualizations to analyze learned representations.

Experimental results

Research questions

• RQ1 Can foundation models pretrained on large-scale, diverse respiratory audio data generalize better than models pretrained on general audio?
• RQ2 How do different self-supervised pretraining objectives (contrastive vs. generative) affect downstream performance in respiratory health tasks?
• RQ3 To what extent can OPERA’s foundation models generalize to unseen datasets and new respiratory audio modalities?
• RQ4 How does fine-tuning improve performance compared to linear probing in respiratory audio classification and regression tasks?
• RQ5 Can an open, reproducible benchmark system accelerate research in respiratory acoustic modeling?

Key findings

• OPERA’s foundation models outperform general-purpose audio models (AudioMAE, CLAP) on 16 out of 19 downstream tasks, demonstrating the advantage of domain-specific pretraining.
• The generative pretraining model (OPERA-GT) achieved the best mean reciprocal rank (MRR) of 0.762 on health condition inference tasks, outperforming both contrastive models and baselines.
• On lung function estimation tasks, OPERA-CE achieved the lowest MAPE of 0.161 ± 0.152 on FEV1/FVC (breath) and 0.193 ± 0.065 on breathing rate, indicating strong regression performance.
• Fine-tuning significantly improved performance: for COPD detection (T7), AUROC increased from 0.855 (linear) to 0.957 (fine-tuned) for OPERA-CT, and to 0.986 for OPERA-GT.
• T-SNE visualizations confirmed that OPERA-CE learned discriminative, semantically meaningful representations, with clear clustering of different respiratory conditions.
• The model fine-tuned on snoring-based body position recognition (T12) achieved an AUROC of 0.994 with OPERA-CT, outperforming all baselines and showing strong generalization to new tasks.