[论文解读] Generative Adversarial Networks for Image-to-Image Translation on Multi-Contrast MR Images - A Comparison of CycleGAN and UNIT

Per Welander, Simon Karlsson+1 more|arXiv (Cornell University)|Jun 20, 2018

Generative Adversarial Networks and Image Synthesis参考文献 10被引用 84

一句话总结

本文比较 CycleGAN 和 UNIT（以及有监督变体）在使用 2D 层切片进行 T1 加权与 T2 加权 MR 图像之间的翻译，评估定量指标和感知真实度，并指出在视觉上逼真的图像并不总是与定量误差度量一致。

ABSTRACT

In medical imaging, a general problem is that it is costly and time consuming to collect high quality data from healthy and diseased subjects. Generative adversarial networks (GANs) is a deep learning method that has been developed for synthesizing data. GANs can thereby be used to generate more realistic training data, to improve classification performance of machine learning algorithms. Another application of GANs is image-to-image translations, e.g. generating magnetic resonance (MR) images from computed tomography (CT) images, which can be used to obtain multimodal datasets from a single modality. Here, we evaluate two unsupervised GAN models (CycleGAN and UNIT) for image-to-image translation of T1- and T2-weighted MR images, by comparing generated synthetic MR images to ground truth images. We also evaluate two supervised models; a modification of CycleGAN and a pure generator model. A small perceptual study was also performed to evaluate how visually realistic the synthesized images are. It is shown that the implemented GAN models can synthesize visually realistic MR images (incorrectly labeled as real by a human). It is also shown that models producing more visually realistic synthetic images not necessarily have better quantitative error measurements, when compared to ground truth data. Code is available at https://github.com/simontomaskarlsson/GAN-MRI

研究动机与目标

• Motivate the use of GANs to synthesize MR data and enable multimodal datasets for improved medical imaging tasks.
• Evaluate unsupervised CycleGAN and UNIT for image-to-image translation between T1- and T2-weighted MR images.
• Investigate supervised variants of CycleGAN and a generator-only model to understand the role of ground-truth guidance.
• Assess visual realism of synthetic MR images via a perceptual study.
• Provide public code to reproduce the experiments.

提出的方法

• Implement CycleGAN and UNIT in Keras for unpaired image-to-image translation between MR contrasts.
• Create two CycleGAN variants: CycleGAN_s (supervised with MAE) and Generators_s (generator-only with MAE).
• Include a simple two-layer CNN baseline as a control.
• Train on 2D axial slices from the Human Connectome Project dataset with 900 training and 213 testing images per domain.
• Evaluate with quantitative metrics (MAE, PSNR, MI) after normalizing images, and with a perceptual study to assess realism.
• Compare generation speed and training times per epoch (CycleGAN vs UNIT) and report qualitative results via visual differences.

实验结果

研究问题

• RQ1 Which GAN model (CycleGAN vs UNIT) best translates MR images between T1 and T2 domains under unsupervised learning?
• RQ2 Do supervised training variants (CycleGAN_s, Generators_s) improve quantitative metrics, and at what cost to visual realism?
• RQ3 How do qualitative perceptual assessments of realism compare with quantitative error metrics across models?
• RQ4 What is the impact of using adversarial loss versus purely supervised MAE loss on image quality and edge detail?
• RQ5 Can these models provide a basis for data augmentation or multimodal MRI data synthesis?

主要发现

• The Generators_s model achieves the best performance across quantitative metrics (MAE, PSNR, MI).
• The Simple baseline model performs worst on quantitative metrics.
• CycleGAN, CycleGAN_s, and UNIT show similar quantitative performance, with T1 images generally scoring better than T2 in MAE.
• Perceptual study indicates synthetic MR images can be visually realistic (often labeled as real), with T2 images being harder to classify as synthetic.
• Supervised training (MAE) yields visually smooth, sometimes unrealistic images, suggesting adversarial loss is important for realism.
• Edges and CSF/white-matter boundaries present more challenges, particularly in T1 and T2 translations, respectively.