[论文解读] CyTran: A Cycle-Consistent Transformer with Multi-Level Consistency for Non-Contrast to Contrast CT Translation

Nicolae-Cătălin Ristea, Andreea-Iuliana Ionescu+5 more|arXiv (Cornell University)|Oct 12, 2021

Brain Tumor Detection and Classification被引用 8

一句话总结

CyTran 提出一种具有多级一致性的循环一致性变换器，用于无增强CT到增强CT的配对缺失翻译，采用混合卷积-变换器架构和多级循环一致性损失，生成高保真、无伪影的CT扫描。在翻译质量和下游图像配准方面均优于最先进方法，人工评估证实未引入视觉伪影。

ABSTRACT

We propose a novel approach to translate unpaired contrast computed tomography (CT) scans to non-contrast CT scans and the other way around. Solving this task has two important applications: (i) to automatically generate contrast CT scans for patients for whom injecting contrast substance is not an option, and (ii) to enhance the alignment between contrast and non-contrast CT by reducing the differences induced by the contrast substance before registration. Our approach is based on cycle-consistent generative adversarial convolutional transformers, for short, CyTran. Our neural model can be trained on unpaired images, due to the integration of a multi-level cycle-consistency loss. Aside from the standard cycle-consistency loss applied at the image level, we propose to apply additional cycle-consistency losses between intermediate feature representations, which enforces the model to be cycle-consistent at multiple representations levels, leading to superior results. To deal with high-resolution images, we design a hybrid architecture based on convolutional and multi-head attention layers. In addition, we introduce a novel data set, Coltea-Lung-CT-100W, containing 100 3D triphasic lung CT scans (with a total of 37,290 images) collected from 100 female patients (there is one examination per patient). Each scan contains three phases (non-contrast, early portal venous, and late arterial), allowing us to perform experiments to compare our novel approach with state-of-the-art methods for image style transfer. Our empirical results show that CyTran outperforms all competing methods. Moreover, we show that CyTran can be employed as a preliminary step to improve a state-of-the-art medical image alignment method. We release our novel model and data set as open source at https://github.com/ristea/cycle-transformer.

研究动机与目标

• 为无法接受对比剂的患者解决从无增强CT生成增强CT扫描的挑战。
• 减少增强CT与无增强CT扫描之间Hounsfield单位（HU）的差异，以提高图像配准精度。
• 开发一种鲁棒、无伪影的翻译模型，以在医学影像中保持解剖结构保真度。
• 引入一种新型大规模三相肺部CT数据集（Coltea-Lung-CT-100W），用于训练和基准测试。
• 证明CyTran可提升最先进医学图像配准流水线的性能。

提出的方法

• 提出一种结合卷积层与多头自注意力机制的混合神经架构，以高效处理高分辨率3D CT扫描。
• 采用多级循环一致性（MLCC）损失，不仅在图像层面，也在中间特征表示层面强制一致性。
• 使用循环一致的生成对抗网络（CycleGAN风格）框架，在无配对数据上进行训练，实现无增强CT与增强CT之间的双向翻译。
• 采用递归级联细化策略，结合ViT-V-Net与CyTran，逐步提升翻译质量。
• 利用一种新型3D三相CT数据集（Coltea-Lung-CT-100W），包含100名女性患者和37,290张图像，用于训练和评估。
• 将CyTran模型作为预处理步骤，用于在图像配准前减少HU差异，从而提升对齐性能。

实验结果

研究问题

• RQ1 基于变换器的模型若结合多级循环一致性，是否能在无增强CT到增强CT翻译方面优于现有基于GAN的方法？
• RQ2 引入多级循环一致性损失是否能提升翻译保真度并减少医学图像翻译中的视觉伪影？
• RQ3 CyTran能否有效减少增强CT与无增强CT扫描之间的Hounsfield单位（HU）差异，以提升下游图像配准性能？
• RQ4 所提出的混合CNN-Transformer架构是否能够生成全分辨率、高保真的CT扫描，且无模式崩溃或结构伪影？
• RQ5 将CyTran集成到配准流水线中，与标准配准方法相比，能在多大程度上提升对齐精度？

主要发现

• CyTran在图像翻译和配准任务中均显著优于最先进方法，包括pix2pix、U-GAT-IT、CycleGAN、CWT-GAN和AttentionGAN。
• 多级循环一致性（MLCC）损失相比标准循环一致性，带来更优的配准性能，证实其在多级特征层面保持结构一致性的有效性。
• 人工评估确认，CyTran是唯一在翻译过程中未引入视觉伪影的方法，这在临床应用中具有关键优势。
• 递归级联的ViT-V-Net与CyTran结合实现了最高的配准性能，动脉相→原始相翻译在3级级联时达到最优，静脉相→原始相翻译在2级级联时达到最优。
• CyTran通过减少HU水平差异，提升了最先进图像配准方法的性能，使增强CT与无增强CT扫描的对齐更加精确。
• 发布Coltea-Lung-CT-100W数据集，包含100例三相3D肺部CT扫描（37,290张图像），为未来医学图像翻译与配准研究提供了宝贵基准。