[Paper Review] Contrastive Learning for Local and Global Learning MRI Reconstruction

Qiaosi Yi, Jinhao Liu+3 more|arXiv (Cornell University)|Nov 30, 2021

Advanced MRI Techniques and Applications47 references5 citations

TL;DR

This paper proposes CLGNet, a novel MRI reconstruction network that integrates spatial and Fourier domain learning via a Spatial and Fourier Layer (SFL) to capture both local and global image structures, while using contrastive learning to constrain the solution space by pulling reconstructions toward fully sampled images and pushing them away from undersampled ones. The method achieves state-of-the-art performance across multiple datasets and acceleration rates.

ABSTRACT

Magnetic Resonance Imaging (MRI) is an important medical imaging modality, while it requires a long acquisition time. To reduce the acquisition time, various methods have been proposed. However, these methods failed to reconstruct images with a clear structure for two main reasons. Firstly, similar patches widely exist in MR images, while most previous deep learning-based methods ignore this property and only adopt CNN to learn local information. Secondly, the existing methods only use clear images to constrain the upper bound of the solution space, while the lower bound is not constrained, so that a better parameter of the network cannot be obtained. To address these problems, we propose a Contrastive Learning for Local and Global Learning MRI Reconstruction Network (CLGNet). Specifically, according to the Fourier theory, each value in the Fourier domain is calculated from all the values in Spatial domain. Therefore, we propose a Spatial and Fourier Layer (SFL) to simultaneously learn the local and global information in Spatial and Fourier domains. Moreover, compared with self-attention and transformer, the SFL has a stronger learning ability and can achieve better performance in less time. Based on the SFL, we design a Spatial and Fourier Residual block as the main component of our model. Meanwhile, to constrain the lower bound and upper bound of the solution space, we introduce contrastive learning, which can pull the result closer to the clear image and push the result further away from the undersampled image. Extensive experimental results on different datasets and acceleration rates demonstrate that the proposed CLGNet achieves new state-of-the-art results.

Motivation & Objective

• To address the limitation of existing deep learning methods in reconstructing MRI images with clear structural details due to insufficient modeling of global context and similar patches.
• To overcome the ill-posed nature of MRI reconstruction by constraining both the upper and lower bounds of the solution space.
• To develop an efficient alternative to self-attention and transformers for global feature learning with reduced computational cost.
• To improve image reconstruction quality by jointly learning in spatial and wavelet domains.

Proposed method

• Proposes a Spatial and Fourier Layer (SFL) that combines standard convolution with Fast Fourier Convolution to jointly learn local features in the spatial domain and global dependencies in the Fourier domain.
• Designs a Spatial and Fourier Residual Block (SFRB) and its residual variant (SFRIR) as the core building blocks of CLGNet, enabling effective feature learning across domains.
• Introduces a contrastive learning loss that pulls the network output closer to the fully sampled (clear) image and pushes it away from the undersampled input, thereby constraining both upper and lower bounds of the solution space.
• Employs a dual-branch architecture with a spatial branch and a wavelet branch to enhance structural detail recovery by leveraging sparsity in the wavelet domain.
• Uses Fast Fourier Transform (FFT) and Inverse FFT (IFFT) within SFL to enable global receptive fields without the high computational cost of self-attention or transformers.
• Maintains model efficiency by keeping the number of parameters and FLOPs low, with only a marginal increase compared to standard convolutional networks.

Experimental results

Research questions

• RQ1 Can joint learning in spatial and Fourier domains improve MRI reconstruction quality by capturing both local and global image structures?
• RQ2 Does the proposed SFL layer outperform self-attention and transformers in learning global context with lower computational cost?
• RQ3 To what extent does contrastive learning improve reconstruction by constraining both upper and lower bounds of the solution space?
• RQ4 How effective is the wavelet branch in enhancing structural detail recovery in MRI reconstruction?
• RQ5 Can the proposed CLGNet achieve state-of-the-art performance across diverse datasets and acceleration rates?

Key findings

• CLGNet achieves new state-of-the-art performance on multiple datasets, including CC-359 Brain, with significant improvements in PSNR and SSIM across various acceleration rates.
• The ablation study shows that removing the wavelet branch leads to a notable performance drop, confirming its contribution to structural detail recovery.
• Replacing SFL with self-attention or transformer layers results in worse performance and significantly higher running times, demonstrating SFL’s superior efficiency and effectiveness.
• The contrastive learning loss improves reconstruction quality by 4 dB in PSNR compared to the baseline without it, proving its effectiveness in solution space constraint.
• Removing negative samples in contrastive learning leads to degraded performance, confirming that constraining the lower bound is essential for finding better solutions.
• The SFL module increases running time and FLOPs only slightly compared to standard convolutions, indicating its computational efficiency.