[Paper Review] Unsupervised Multi-Modal Medical Image Registration via Discriminator-Free Image-to-Image Translation

Zekang Chen, Jia Wei+1 more|arXiv (Cornell University)|Apr 28, 2022

Advanced Neural Network Applications4 citations

TL;DR

This paper proposes a discriminator-free, shape-preserving image-to-image translation framework for unsupervised multi-modal medical image registration. By replacing adversarial training with a pixel loss and a patchwise contrastive loss (PatchNCE), the method improves deformation field accuracy and reduces artifacts, achieving state-of-the-art performance on Learn2Reg 2021 datasets with DSC scores up to 0.966 and HD95 of 4.367.

ABSTRACT

In clinical practice, well-aligned multi-modal images, such as Magnetic Resonance (MR) and Computed Tomography (CT), together can provide complementary information for image-guided therapies. Multi-modal image registration is essential for the accurate alignment of these multi-modal images. However, it remains a very challenging task due to complicated and unknown spatial correspondence between different modalities. In this paper, we propose a novel translation-based unsupervised deformable image registration approach to convert the multi-modal registration problem to a mono-modal one. Specifically, our approach incorporates a discriminator-free translation network to facilitate the training of the registration network and a patchwise contrastive loss to encourage the translation network to preserve object shapes. Furthermore, we propose to replace an adversarial loss, that is widely used in previous multi-modal image registration methods, with a pixel loss in order to integrate the output of translation into the target modality. This leads to an unsupervised method requiring no ground-truth deformation or pairs of aligned images for training. We evaluate four variants of our approach on the public Learn2Reg 2021 datasets \cite{hering2021learn2reg}. The experimental results demonstrate that the proposed architecture achieves state-of-the-art performance. Our code is available at https://github.com/heyblackC/DFMIR.

Motivation & Objective

• To address the challenge of inaccurate and inconsistent image translations in GAN-based multi-modal registration, which introduce artifacts due to discriminator misinterpretation of domain-specific deformations.
• To improve unsupervised multi-modal registration by transforming the problem into a mono-modal registration task via image translation, avoiding reliance on ground-truth deformation fields.
• To enhance shape consistency and appearance fidelity in cross-modality translation using a contrastive loss and a pixel loss, replacing adversarial training.
• To achieve state-of-the-art registration accuracy without using discriminators, which are shown to degrade performance by encouraging spurious deformations.
• To validate the method on public datasets and ablate key components, including the role of local and global losses and the impact of the discriminator.

Proposed method

• Replaces the standard GAN-based image-to-image translation with a discriminator-free generator network to avoid introducing artifacts from domain-specific deformations.
• Employs a patchwise contrastive loss (PatchNCE) to preserve structural and shape consistency during translation between modalities.
• Uses a pixel-wise reconstruction loss to align the translated image with the target modality’s appearance, enabling the use of mono-modal similarity metrics.
• Integrates both local (PatchNCE) and global (pixel loss) alignment losses to guide the registration network toward accurate deformation fields.
• Trains the registration network end-to-end using only image similarity, without requiring ground-truth deformation fields or paired aligned images.
• Applies a two-stream architecture: one for translation (source to target modality) and one for deformation field prediction, with shared feature learning.

Experimental results

Research questions

• RQ1 Can removing the discriminator from GAN-based image translation improve shape consistency and reduce artifacts in multi-modal medical image registration?
• RQ2 Does a contrastive loss (PatchNCE) effectively preserve anatomical structure during cross-modality translation without adversarial supervision?
• RQ3 Can a pixel loss serve as an effective similarity metric for training a registration network when combined with appearance transfer in unsupervised settings?
• RQ4 How do local and global alignment losses jointly contribute to improved registration accuracy in the absence of ground-truth labels?
• RQ5 Does the absence of a discriminator lead to better generalization and performance compared to GAN-based baselines on real medical imaging data?

Key findings

• The proposed method achieves state-of-the-art performance on the Learn2Reg 2021 dataset, with a DSC of 0.938 (±0.025) and HD95 of 7.737 mm (±1.791) for the Insp-Exp dataset.
• On the Insp-Exp dataset, the method achieves a DSC of 0.966 (±0.019) and HD95 of 4.367 mm (±1.048), outperforming all baselines including SbR and RGPT.
• Ablation studies show that removing either the local (PatchNCE) or global (pixel) loss leads to performance degradation, confirming their complementary roles.
• The variant with a discriminator (w/ dis) performs worse than the proposed method, demonstrating that discriminators introduce harmful artifacts and degrade registration accuracy.
• Visualizations confirm that the proposed method produces smoother deformation fields and better boundary alignment than RGPT and SbR, especially in regions with large deformations.
• The method successfully reduces noise and jitter in deformation fields compared to RGPT, while avoiding the failure of SbR in handling large-scale anatomical changes.