[Paper Review] Multi-Contrast MRI Super-Resolution via a Multi-Stage Integration Network

Chun-Mei Feng, Huazhu Fu+2 more|arXiv (Cornell University)|May 19, 2021

Advanced Image Processing Techniques26 references5 citations

TL;DR

This paper proposes MINet, a multi-stage integration network for multi-contrast MRI super-resolution that explicitly models cross-contrast dependencies across hierarchical feature stages. By fusing complementary features from auxiliary contrasts using similarity-based integration and attention mechanisms, MINet achieves state-of-the-art performance on fastMRI and clinical datasets, significantly improving PSNR and SSIM over prior methods.

ABSTRACT

Super-resolution (SR) plays a crucial role in improving the image quality of magnetic resonance imaging (MRI). MRI produces multi-contrast images and can provide a clear display of soft tissues. However, current super-resolution methods only employ a single contrast, or use a simple multi-contrast fusion mechanism, ignoring the rich relations among different contrasts, which are valuable for improving SR. In this work, we propose a multi-stage integration network (i.e., MINet) for multi-contrast MRI SR, which explicitly models the dependencies between multi-contrast images at different stages to guide image SR. In particular, our MINet first learns a hierarchical feature representation from multiple convolutional stages for each of different-contrast image. Subsequently, we introduce a multi-stage integration module to mine the comprehensive relations between the representations of the multi-contrast images. Specifically, the module matches each representation with all other features, which are integrated in terms of their similarities to obtain an enriched representation. Extensive experiments on fastMRI and real-world clinical datasets demonstrate that 1) our MINet outperforms state-of-the-art multi-contrast SR methods in terms of various metrics and 2) our multi-stage integration module is able to excavate complex interactions among multi-contrast features at different stages, leading to improved target-image quality.

Motivation & Objective

• Address the limitation of existing super-resolution methods that ignore complex inter-contrast relationships in multi-contrast MRI.
• Leverage complementary information from multiple MRI contrasts (e.g., T1WI, T2WI, PDWI, FS-PDWI) to improve target-contrast image restoration.
• Develop a deep learning framework that models hierarchical feature interactions across different stages to enhance representation learning.
• Improve image quality by integrating multi-contrast features at multiple levels using similarity-aware fusion and attention mechanisms.
• Demonstrate robust performance across diverse datasets and scaling factors, including challenging 4× super-resolution.

Proposed method

• Proposes a multi-stage integration network (MINet) that processes each MRI contrast through a shared encoder with multiple convolutional stages to learn hierarchical features.
• Introduces a multi-stage integration module that matches features from different contrasts based on their similarity and fuses them into enriched representations.
• Employs a feature attention module ($\mathcal{F}_{\text{Att}}$) to adaptively emphasize informative features within each contrast, enhancing modality-specific representation.
• Applies a feature integration module ($\mathcal{F}_{\text{Int}}$) that aggregates features across contrasts at each stage, modeling cross-contrast dependencies through weighted fusion.
• Uses a progressive refinement strategy where features from earlier stages guide the learning of deeper representations, enabling multi-level interaction.
• Trains the network using a composite loss function combining L1 and perceptual losses to preserve structural details and improve visual quality.

Experimental results

Research questions

• RQ1 Can modeling cross-contrast dependencies at multiple hierarchical stages improve super-resolution performance in multi-contrast MRI?
• RQ2 How do multi-contrast features from different MRI sequences (e.g., T1WI, T2WI) interact to enhance target-contrast reconstruction?
• RQ3 What is the contribution of stage-wise feature integration versus single-stage fusion in multi-contrast MRI SR?
• RQ4 Does the inclusion of auxiliary contrasts (e.g., T1WI for T2WI SR) lead to measurable improvements in PSNR and SSIM?
• RQ5 How robust is the proposed method across different datasets and scaling factors (e.g., 2×, 4×) in real-world clinical settings?

Key findings

• MINet achieves the highest PSNR (31.769) and SSIM (0.709) on the fastMRI dataset at 2× super-resolution, outperforming the previous SOTA method (Lyu et al.) by 2.286 dB in PSNR.
• On the SMS dataset at 4× super-resolution, MINet achieves a PSNR of 35.032 and SSIM of 0.948, significantly outperforming the second-best method (Lyu et al.) by 1.365 dB in PSNR.
• On the uMR dataset at 4× super-resolution, MINet achieves a PSNR of 34.219 and SSIM of 0.956, surpassing the second-best method (Zeng et al.) by 2.358 dB in PSNR.
• Abllation studies confirm that removing the feature integration module ($\mathcal{F}_{\text{Int}}$) or attention module ($\mathcal{F}_{\text{Att}}$) degrades performance, with the full model yielding the best results (NMSE: 0.0018, PSNR: 40.549, SSIM: 0.983 on SMS at 2×).
• Visual results show MINet produces fewer artifacts (e.g., chessboard effects) and better structural preservation, especially in high-magnification settings.
• The method demonstrates strong generalization across diverse datasets, including fastMRI, SMS, and uMR, confirming robustness in real-world clinical scenarios.