[Paper Review] Frozen Language Model Helps ECG Zero-Shot Learning

Jun Li, Che Liu+3 more|arXiv (Cornell University)|Mar 22, 2023

ECG Monitoring and Analysis9 citations

TL;DR

METS uses a frozen large language model to guide ECG self-supervised pre-training with paired ECG-text data, enabling zero-shot ECG classification and improving performance without annotated data.

ABSTRACT

The electrocardiogram (ECG) is one of the most commonly used non-invasive, convenient medical monitoring tools that assist in the clinical diagnosis of heart diseases. Recently, deep learning (DL) techniques, particularly self-supervised learning (SSL), have demonstrated great potential in the classification of ECG. SSL pre-training has achieved competitive performance with only a small amount of annotated data after fine-tuning. However, current SSL methods rely on the availability of annotated data and are unable to predict labels not existing in fine-tuning datasets. To address this challenge, we propose Multimodal ECG-Text Self-supervised pre-training (METS), the first work to utilize the auto-generated clinical reports to guide ECG SSL pre-training. We use a trainable ECG encoder and a frozen language model to embed paired ECG and automatically machine-generated clinical reports separately. The SSL aims to maximize the similarity between paired ECG and auto-generated report while minimize the similarity between ECG and other reports. In downstream classification tasks, METS achieves around 10% improvement in performance without using any annotated data via zero-shot classification, compared to other supervised and SSL baselines that rely on annotated data. Furthermore, METS achieves the highest recall and F1 scores on the MIT-BIH dataset, despite MIT-BIH containing different classes of ECG compared to the pre-trained dataset. The extensive experiments have demonstrated the advantages of using ECG-Text multimodal self-supervised learning in terms of generalizability, effectiveness, and efficiency.

Motivation & Objective

• Motivate zero-shot learning for ECG to enable classification without annotated data.
• Leverage a multimodal ECG-text setup to inject medical knowledge into ECG representations.
• Develop a contrastive pre-training objective between ECG embeddings and generated clinical reports.
• Show zero-shot classification capability across multiple ECG tasks (diagnostic, form, rhythm).
• Demonstrate cross-dataset generalization to external datasets like MIT-BIH.

Proposed method

• Use a trainable ResNet1d-18 ECG encoder.
• Embed paired ECG and auto-generated reports with a frozen clinical language model.
• Project ECG and text embeddings to a common dimensional space with a shared projection head.
• Optimize a multimodal contrastive loss to maximize ECG-text similarity for true pairs and minimize it for others.
• Perform zero-shot classification by querying the language model with complete-sentence label prompts and measuring ECG-text similarity.

Experimental results

Research questions

• RQ1 Can multimodal ECG-text self-supervised learning enable zero-shot classification without any annotated data?
• RQ2 Does freezing the language model while training the ECG encoder transfer clinical knowledge effectively?
• RQ3 How does METS perform on PTB-XL and MIT-BIH for superclass, form, and rhythm tasks under zero-shot settings?
• RQ4 Is METS robust to cross-dataset transfer without fine-tuning?

Key findings

• METS achieves around 10% improvement in zero-shot performance over supervised and SSL baselines that rely on annotated data.
• On PTB-XL superclass, METS reaches 0.842 accuracy (0% labeled data) with 0.694 precision, 0.626 recall, and 0.657 F1.
• On PTB-XL form, METS reaches 0.734 accuracy (0% labeled data) with 0.537 precision, 0.503 recall, and 0.518 F1.
• On PTB-XL rhythm, METS reaches 0.746 accuracy (0% labeled data) with 0.576 precision, 0.612 recall, and 0.593 F1.
• On MIT-BIH, METS reaches 0.794 accuracy (0% labeled data) with 0.680 precision, 0.735 recall, and 0.706 F1.
• METS outperforms other SSL methods and even surpasses supervised learning in cross-dataset transfers (MIT-BIH).