[Paper Review] Training one model to detect heart and lung sound events from single point auscultations

Leander Melms, Robert R. Ilesan+12 more|arXiv (Cornell University)|Jan 15, 2023

Phonocardiography and Auscultation Techniques4 citations

TL;DR

This paper proposes a semi-supervised multi-task learning approach to train a single deep learning model that detects both heart and lung sound events from single-point auscultations, invariant to auscultation site and device. By combining pseudo-labeled background events from specialist single-task models with ground truth data, the unified model achieves a macro F1 score of 39.2% across six classes—6.7% higher than baseline single-task models—enabling robust, generalized detection of cardiopulmonary sounds from any location.

ABSTRACT

Objective: This work proposes a semi-supervised training approach for detecting lung and heart sounds simultaneously with only one trained model and in invariance to the auscultation point. Methods: We use open-access data from the 2016 Physionet/CinC Challenge, the 2022 George Moody Challenge, and from the lung sound database HF_V1. We first train specialist single-task models using foreground ground truth (GT) labels from different auscultation databases to identify background sound events in the respective lung and heart auscultation databases. The pseudo-labels generated in this way were combined with the ground truth labels in a new training iteration, such that a new model was subsequently trained to detect foreground and background signals. Benchmark tests ensured that the newly trained model could detect both, lung, and heart sound events in different auscultation sites without regressing on the original task. We also established hand-validated labels for the respective background signal in heart and lung sound auscultations to evaluate the models. Results: In this work, we report for the first time results for i) a multi-class prediction for lung sound events and ii) for simultaneous detection of heart and lung sound events and achieve competitive results using only one model. The combined multi-task model regressed slightly in heart sound detection and gained significantly in lung sound detection accuracy with an overall macro F1 score of 39.2% over six classes, representing a 6.7% improvement over the single-task baseline models. Conclusion/Significance: To the best of our knowledge, this is the first approach developed to date for measuring heart and lung sound events invariant to both, the auscultation site and capturing device. Hence, our model is capable of performing lung and heart sound detection from any auscultation location.

Motivation & Objective

• To develop a unified model capable of detecting both heart and lung sound events from single-point auscultations, regardless of location or device.
• To address the challenge of limited annotated data for multi-modality lung and heart sound detection.
• To improve detection performance by leveraging pseudo-labels from specialist models to enhance generalization.
• To achieve invariance to auscultation site and recording device in a single, end-to-end trainable model.
• To establish a benchmark for simultaneous, multi-class detection of lung and heart sound events using minimal human-annotated data.

Proposed method

• The method begins with training separate single-task models on foreground ground truth labels from the 2016 PhysioNet/CinC Challenge, 2022 George Moody Challenge, and HF_V1 lung sound database.
• Background sound events are identified using these trained models, and their predictions are used as pseudo-labels for data not labeled in the original datasets.
• Pseudo-labeled background data is combined with original ground truth data to form a new, expanded training set for a joint multi-task model.
• The joint model is trained to detect both foreground (abnormal) and background (normal) events in both heart and lung sound recordings.
• A hand-validated dataset of background signals in heart and lung auscultations is created to evaluate model performance independently.
• The final model is benchmarked on its ability to detect events across different auscultation sites and devices, ensuring invariance.

Experimental results

Research questions

• RQ1 Can a single deep learning model detect both heart and lung sound events with high accuracy across different auscultation sites and devices?
• RQ2 How does semi-supervised pseudo-labeling of background events improve detection performance in low-resource multi-modal auscultation data?
• RQ3 To what extent does joint multi-task training degrade performance on individual tasks compared to single-task models?
• RQ4 Can the model generalize to new auscultation locations without fine-tuning or retraining?
• RQ5 What is the impact of combining pseudo-labeled background data with real ground truth on overall detection F1 score?

Key findings

• The combined multi-task model achieved a macro F1 score of 39.2% across six lung and heart sound event classes, representing a 6.7% improvement over single-task baseline models.
• The model showed significant gains in lung sound detection accuracy, while maintaining acceptable performance in heart sound detection.
• The model demonstrated invariance to auscultation site and recording device, enabling reliable detection from any single-point location.
• Pseudo-labeling of background events from specialist models effectively enhanced the generalization of the joint model.
• Hand-validated background labels confirmed the model's ability to correctly identify normal physiological sounds in both heart and lung recordings.
• Benchmarking confirmed that the model did not regress on the original single-task performance, validating the effectiveness of the semi-supervised multi-task approach.