[论文解读] Breaking the Dilemma of Medical Image-to-image Translation

Lingke Kong, Chenyu Lian+4 more|arXiv (Cornell University)|Oct 13, 2021

Radiomics and Machine Learning in Medical Imaging参考文献 24被引用 74

一句话总结

RegGAN 引入了一种带有注册网络的损失校正框架，用于处理医学图像翻译中的错位目标，在对齐、错位和未配对数据上胜过 Pix2Pix 和 Cycle-consistency。

ABSTRACT

Supervised Pix2Pix and unsupervised Cycle-consistency are two modes that dominate the field of medical image-to-image translation. However, neither modes are ideal. The Pix2Pix mode has excellent performance. But it requires paired and well pixel-wise aligned images, which may not always be achievable due to respiratory motion or anatomy change between times that paired images are acquired. The Cycle-consistency mode is less stringent with training data and works well on unpaired or misaligned images. But its performance may not be optimal. In order to break the dilemma of the existing modes, we propose a new unsupervised mode called RegGAN for medical image-to-image translation. It is based on the theory of "loss-correction". In RegGAN, the misaligned target images are considered as noisy labels and the generator is trained with an additional registration network to fit the misaligned noise distribution adaptively. The goal is to search for the common optimal solution to both image-to-image translation and registration tasks. We incorporated RegGAN into a few state-of-the-art image-to-image translation methods and demonstrated that RegGAN could be easily combined with these methods to improve their performances. Such as a simple CycleGAN in our mode surpasses latest NICEGAN even though using less network parameters. Based on our results, RegGAN outperformed both Pix2Pix on aligned data and Cycle-consistency on misaligned or unpaired data. RegGAN is insensitive to noises which makes it a better choice for a wide range of scenarios, especially for medical image-to-image translation tasks in which well pixel-wise aligned data are not available

研究动机与目标

• 动机：需要一种能够处理错位或非配对的医学图像的图像到图像翻译模式，超越 Pix2Pix 和 Cycle-consistency。
• 提出 RegGAN 作为一种损失校正框架，联合训练生成器和注册网络，以自适应建模形变噪声。
• 证明 RegGAN 能与现有翻译模型集成，在更少参数下提升性能。
• 在公开的医学影像数据集上，在对齐、错位和非配对条件以及不同噪声水平下评估 RegGAN。

提出的方法

• 在目标图像错位时，将图像到图像翻译建模为带有嘈杂标签的监督学习。
• 在生成器 G 之后引入一个注册网络 R，以学习一个形变场来纠正标签噪声。
• 定义一个校正损失，最小化错位目标与扭曲后的生成器输出之间的差异，以及形变场的平滑性损失。
• 将 Corr 损失、Smooth 损失和对抗损失组合成总损失，用于对 G、R 以及 D 的联合优化。
• 将形变噪声视为噪声模型；使用损失校正将嘈杂的标签与真实目标分布对齐。

实验结果

研究问题

• RQ1 当目标图像错位或非配对时，RegGAN 是否能在翻译质量上与 Pix2Pix 和 Cycle-consistency 相竞争甚至更优？
• RQ2 将注册网络引入以建模形变噪声，是否提升对错位和不同噪声水平的鲁棒性？
• RQ3 RegGAN 是否可以与现有的图像到图像翻译架构有效集成，以在更少参数下提升性能？
• RQ4 在多数据集和多方法下，RegGAN 在对齐数据与错位/非配对数据上的表现如何？

主要发现

• 在目标错位或非配对的情况下，RegGAN 在评估方法中始终提升翻译性能。
• 引入注册网络 (+R) 在各指标和方法上带来显著的性能提升，往往超过 Cycle-consistency 基线。
• NC+R 配置（非循环一致性加注册）常常优于 C+R 配置，表明循环一致性在与 RegGAN 结合时可能阻碍性能。
• RegGAN 对增加的噪声水平和非仿射噪声仍然具有鲁棒性，指标持续优于 Pix2Pix 和 CycleGAN 基线。
• 在非配对数据上，RegGAN 的表现优于 Pix2Pix 和 CycleGAN，尽管增益不如配对情形明显。
• 该方法可以与不同架构（CycleGAN、MUNIT、UNIT、NICEGAN）集成，并在参数更少的情况下取得更好或可比的结果。