[Paper Review] Aligning Multi-Sequence CMR Towards Fully Automated Myocardial Pathology Segmentation

Wangbin Ding, Lei Li+6 more|arXiv (Cornell University)|Feb 7, 2023

Medical Imaging Techniques and ApplicationsMedicine3 citations

TL;DR

This paper proposes U-MyoPS, a fully automatic framework for myocardial pathology segmentation in unaligned multi-sequence cardiac MRI (MS-CMR) by jointly performing image registration and feature fusion. It introduces a novel multi-sequence fusion module and a structure propagation head to enhance segmentation accuracy using spatial relationships between pathology and myocardium, achieving state-of-the-art performance on both private and public MS-CMR datasets without requiring pre-alignment.

ABSTRACT

Myocardial pathology segmentation (MyoPS) is critical for the risk stratification and treatment planning of myocardial infarction (MI). Multi-sequence cardiac magnetic resonance (MS-CMR) images can provide valuable information. For instance, balanced steady-state free precession cine sequences present clear anatomical boundaries, while late gadolinium enhancement and T2-weighted CMR sequences visualize myocardial scar and edema of MI, respectively. Existing methods usually fuse anatomical and pathological information from different CMR sequences for MyoPS, but assume that these images have been spatially aligned. However, MS-CMR images are usually unaligned due to the respiratory motions in clinical practices, which poses additional challenges for MyoPS. This work presents an automatic MyoPS framework for unaligned MS-CMR images. Specifically, we design a combined computing model for simultaneous image registration and information fusion, which aggregates multi-sequence features into a common space to extract anatomical structures (i.e., myocardium). Consequently, we can highlight the informative regions in the common space via the extracted myocardium to improve MyoPS performance, considering the spatial relationship between myocardial pathologies and myocardium. Experiments on a private MS-CMR dataset and a public dataset from the MYOPS2020 challenge show that our framework could achieve promising performance for fully automatic MyoPS.

Motivation & Objective

• To address the challenge of myocardial pathology segmentation in clinically unaligned multi-sequence cardiac MRI (MS-CMR) due to respiratory motion.
• To develop a unified framework that simultaneously performs image registration and feature fusion for unaligned MS-CMR scans.
• To improve segmentation performance by leveraging the spatial relationship between myocardial pathology and the myocardium structure.
• To enable fully automatic, time-efficient, and clinically applicable pathology segmentation without requiring pre-aligned data.

Proposed method

• Proposes a combined computing model that jointly optimizes multi-sequence image registration and feature fusion in an end-to-end manner.
• Introduces a Multi-Sequence Fusion (MSF) module to aggregate spatially unaligned feature maps from different CMR sequences into a common anatomical space.
• Employs a Structure Propagation Head (SPH) to propagate extracted myocardial contours to enhance the segmentation of scar and edema regions.
• Uses a U-Net-based architecture with skip connections to preserve spatial details and improve segmentation accuracy.
• Trains the model using a multi-task loss combining registration, myocardium segmentation, and pathology segmentation objectives.
• Applies a two-stage training strategy: first pre-train registration and myocardium extraction, then fine-tune with pathology segmentation.

Figure 1: Visualization of unaligned and aligned multi-sequence cardiac magnetic resonance (MS-CMR) images. (a) Short-axis views of unaligned MS-CMR images, i.e., bSSFP (top), LGE (middle) and T2 (bottom), with contours of the left ventricle (LV) epicardium, scar and edema regions. (b) Warped bSSFP

Experimental results

Research questions

• RQ1 Can a unified deep learning framework effectively perform joint registration and segmentation on unaligned multi-sequence cardiac MRI?
• RQ2 How does incorporating myocardial structure information improve the accuracy of myocardial pathology segmentation?
• RQ3 What is the impact of spatial misalignment between bSSFP, LGE, and T2-weighted CMR sequences on pathology segmentation performance?
• RQ4 Can a joint optimization strategy of registration, myocardium extraction, and pathology segmentation outperform independent training?
• RQ5 How does the proposed method generalize across different acquisition protocols and datasets?

Key findings

• U-MyoPS achieved a Dice score of 73.1 on the private unaligned MS-CMR dataset, demonstrating strong performance without requiring pre-alignment.
• On the MYOPS2020 public dataset, U-MyoPS achieved a Dice score of 68.6, outperforming most existing methods and ranking among the top in the challenge.
• The method reduced the negative impact of spatial misalignment by jointly registering and fusing multi-sequence features, improving segmentation consistency.
• The Structure Propagation Head significantly enhanced pathology segmentation by leveraging the spatial relationship between myocardium and lesions.
• The inference time was only 10 seconds per study, confirming the method’s time efficiency for clinical deployment.
• Despite strong performance, the method showed performance degradation when tested on data with significantly different acquisition protocols, indicating a need for improved generalization.

Figure 2: Pipeline of the myocardial pathology segmentation (MyoPS) framework, referred to as U-MyoPS, for unaligned multi-sequence cardiac magnetic resonance (MS-CMR) images. The network architecture of the U-MyoPS includes three encoders ( $E_{{bSSFP}}$ , $E_{LGE}$ and $E_{T2}$ ) and two registrat