[论文解读] Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network

Zizhao Zhang, Lin Yang+1 more|arXiv (Cornell University)|Feb 27, 2018

Generative Adversarial Networks and Image Synthesis参考文献 42被引用 53

一句话总结

提出一个三维端到端GAN框架，联合跨模态医学体积翻译（CT/MRI），通过加入循环一致性和形状一致性损失以及在线合成数据增强来提高分割。

ABSTRACT

Synthesized medical images have several important applications, e.g., as an intermedium in cross-modality image registration and as supplementary training samples to boost the generalization capability of a classifier. Especially, synthesized computed tomography (CT) data can provide X-ray attenuation map for radiation therapy planning. In this work, we propose a generic cross-modality synthesis approach with the following targets: 1) synthesizing realistic looking 3D images using unpaired training data, 2) ensuring consistent anatomical structures, which could be changed by geometric distortion in cross-modality synthesis and 3) improving volume segmentation by using synthetic data for modalities with limited training samples. We show that these goals can be achieved with an end-to-end 3D convolutional neural network (CNN) composed of mutually-beneficial generators and segmentors for image synthesis and segmentation tasks. The generators are trained with an adversarial loss, a cycle-consistency loss, and also a shape-consistency loss, which is supervised by segmentors, to reduce the geometric distortion. From the segmentation view, the segmentors are boosted by synthetic data from generators in an online manner. Generators and segmentors prompt each other alternatively in an end-to-end training fashion. With extensive experiments on a dataset including a total of 4,496 CT and magnetic resonance imaging (MRI) cardiovascular volumes, we show both tasks are beneficial to each other and coupling these two tasks results in better performance than solving them exclusively.

研究动机与目标

• 解决 CT 与 MRI 之间翻译缺乏成对的跨模态医学体积的问题。
• 开发一个3D GAN框架，通过形状一致性在翻译过程中保留解剖形状。
• 联合训练生成器和分割器，以利用合成数据提升分割性能。
• 证明在线使用合成数据在翻译质量和分割上优于离线增强。
• 在大型心血管 CT/MRI 数据集上验证该方法，展示翻译与分割的互利。

提出的方法

• 使用两个生成器 G_A 和 G_B 在跨域体积翻译中配合判别器 D_A 和 D_B。
• 应用循环一致性损失，确保 G_A(G_B(x_A)) ≈ x_A 且 G_B(G_A(x_B)) ≈ x_B。
• 通过分割器 S_A 和 S_B 引入形状一致性损失，将翻译数据映射到共享形状空间 Y，并计算交叉熵损失。
• 用真实数据和在线合成数据共同训练分割器 S_A 和 S_B 以提升分割性能，包括重构的合成数据。
• 将对抗损失、循环一致性损失和形状一致性损失组合成联合目标 L = L_GAN + L_GAN + λ L_cyc + γ L_shape。
• 提供端到端训练，使生成器和分割器相互强化。

实验结果

研究问题

• RQ1 在3D体积中，未配对的 CT 与 MRI 数据是否能跨模态翻译并保持解剖形状？
• RQ2 将形状一致性约束引入是否比仅使用循环一致性提高跨模态翻译质量？
• RQ3 在在线耦合训练框架中生成的合成数据是否比离线增强更能提升分割性能？
• RQ4 联合训练生成器和分割器对翻译质量和分割准确性有何影响？

主要发现

方法	CT Dice (%)	MRI Dice (%)
Baseline (R)	67.8	70.3
ADA (R+S)	66.0	71.0
Ours (R+S)	74.4	73.2

• 形状一致性提升了翻译质量，获得比不采用形状一致性更高的形状质量分数（S-score）。
• 端到端在线使用合成数据显著提升分割性能，相较于基线和离线增强。
• 在包含 4,496 例心血管 CT/MRI 的数据集上，使用合成数据时 CT 和 MRI 分割的 Dice 分数更高（CT 74.4%，MRI 73.2%），相较基线和 ADA 基线。
• 启用形状一致性的生成模型产生的合成体积比 CycleGAN 基线具有更少的伪影和更好的解剖保留。
• 联合训练生成器和分割器的结果总体优于分别解决翻译和分割的做法。