[Paper Review] Multi-Modality Cardiac Image Analysis with Deep Learning

Lei Li, Fuping Wu+2 more|arXiv (Cornell University)|Nov 8, 2021

Cardiac Imaging and Diagnostics4 citations

TL;DR

This paper presents deep learning frameworks for multi-modality cardiac image analysis, focusing on left atrial scar segmentation from LGE MRI using auxiliary non-contrast MRI, and introduces three unsupervised domain adaptation methods—DDFSeg, CFDNet, and VarDA—for cross-modality cardiac segmentation. The proposed methods improve segmentation accuracy by leveraging explicit domain discrepancy metrics and disentangled feature learning, with VarDA achieving state-of-the-art performance on LGE MRI segmentation (Dice: 73.03% for MYO, 88.06% for LV).

ABSTRACT

Accurate cardiac computing, analysis and modeling from multi-modality images are important for the diagnosis and treatment of cardiac disease. Late gadolinium enhancement magnetic resonance imaging (LGE MRI) is a promising technique to visualize and quantify myocardial infarction (MI) and atrial scars. Automating quantification of MI and atrial scars can be challenging due to the low image quality and complex enhancement patterns of LGE MRI. Moreover, compared with the other sequences LGE MRIs with gold standard labels are particularly limited, which represents another obstacle for developing novel algorithms for automatic segmentation and quantification of LGE MRIs. This chapter aims to summarize the state-of-the-art and our recent advanced contributions on deep learning based multi-modality cardiac image analysis. Firstly, we introduce two benchmark works for multi-sequence cardiac MRI based myocardial and pathology segmentation. Secondly, two novel frameworks for left atrial scar segmentation and quantification from LGE MRI were presented. Thirdly, we present three unsupervised domain adaptation techniques for cross-modality cardiac image segmentation.

Motivation & Objective

• To address the challenge of low-quality LGE MRI and limited gold-standard annotations for myocardial infarction and atrial scar quantification.
• To develop robust deep learning frameworks for left atrial scar segmentation using LGE MRI and complementary non-contrast MRI sequences.
• To improve cross-modality cardiac segmentation by introducing unsupervised domain adaptation techniques that reduce domain shift between CT and MR images.
• To evaluate and compare explicit domain discrepancy metrics against adversarial training in segmentation tasks across imaging modalities.

Proposed method

• Proposed LearnGC and AtrialJSQnet networks for left atrial scar segmentation using LGE MRI and non-contrast MRI as auxiliary input.
• Introduced DDFSeg, a domain adaptation framework that disentangles domain-specific features (DSFs) and content features for improved image translation and adaptation.
• Developed CFDNet, which uses a novel explicit metric—concrete Fréchet distance (CFD)—to measure domain discrepancy between source and target features, replacing adversarial training.
• Proposed VarDA, a variational domain adaptation method that uses variational approximation to simplify metric computation and improve feature correlation modeling.
• Applied unsupervised domain adaptation techniques to transfer knowledge from CT to MR images and from MR to LGE MRI, using only unlabeled target data.
• Utilized 2D slice-wise segmentation with multi-sequence MRI inputs, incorporating data augmentation and normalization to improve robustness.

Figure 1: Visualization of myocardial pathology segmentation combing multi-sequence CMR images acquired from the same patient. Image adopted from the website of MyoPS 2020 challenge: http://www.sdspeople.fudan.edu.cn/zhuangxiahai/0/myops20/.

Experimental results

Research questions

• RQ1 Can deep learning models effectively segment left atrial scars in LGE MRI when combined with non-contrast MRI sequences?
• RQ2 How do explicit domain discrepancy metrics compare to adversarial training in cross-modality cardiac image segmentation tasks?
• RQ3 To what extent can feature disentanglement improve domain adaptation performance in multi-modality cardiac MRI analysis?
• RQ4 Can unsupervised domain adaptation frameworks reduce the performance gap between CT and MR cardiac segmentation without requiring labeled target data?
• RQ5 What is the impact of using variational approximation in domain adaptation for improving segmentation accuracy on low-resource LGE MRI data?

Key findings

• VarDA achieved the highest Dice scores on LGE MRI segmentation, with 73.03% for myocardium (MYO), 88.06% for left ventricle (LV), and 78.47% for right ventricle (RV), outperforming CFDNet and SIFA.
• CFDNet achieved comparable segmentation performance to adversarial training methods, with Dice scores of 69.1% (MYO), 86.4% (LV), and 76.0% (RV), demonstrating the effectiveness of explicit metric learning.
• DDFSeg successfully disentangled domain-specific features, improving image translation quality and enabling better domain adaptation in multi-sequence MRI tasks.
• The use of variational approximation in VarDA reduced computational complexity and improved feature correlation modeling, leading to more stable and accurate segmentation.
• PnP-AdaNet and other baselines showed inferior performance compared to VarDA and CFDNet, especially on RV segmentation, highlighting the advantage of explicit metric learning.
• Visual comparisons confirmed that VarDA and CFDNet produced more anatomically plausible segmentation boundaries than baseline methods, particularly in complex regions with low contrast.

Figure 2: The average Dice score of the nine evaluated methods on each slice of the 40 test LGE CMR images. Image adopted from Zhuang et al. [ 28 ]