[論文レビュー] Generative Adversarial Networks for MR-CT Deformable Image Registration

Christine Tanner, Fırat Özdemir+4 more|arXiv (Cornell University)|Jul 19, 2018

Medical Image Segmentation Techniques参考文献 5被引用数 47

ひとこと要約

本論文は cycle-GANベースのMR-CT画像合成を用いた変形的登録を評価し、合成が空間対応を保持することもあれば劣化させることもあると示唆する。腹部の登録は最先端のマルチモーダル法と同等の性能を達成できる一方、胸部の結果は肺容量バイアスにより劣る。

ABSTRACT

Deformable Image Registration (DIR) of MR and CT images is one of the most challenging registration task, due to the inherent structural differences of the modalities and the missing dense ground truth. Recently cycle Generative Adversarial Networks (cycle-GANs) have been used to learn the intensity relationship between these 2 modalities for unpaired brain data. Yet its usefulness for DIR was not assessed. In this study we evaluate the DIR performance for thoracic and abdominal organs after synthesis by cycle-GAN. We show that geometric changes, which differentiate the two populations (e.g. inhale vs. exhale), are readily synthesized as well. This causes substantial problems for any application which relies on spatial correspondences being preserved between the real and the synthesized image (e.g. plan, segmentation, landmark propagation). To alleviate this problem, we investigated reducing the spatial information provided to the discriminator by decreasing the size of its receptive fields. Image synthesis was learned from 17 unpaired subjects per modality. Registration performance was evaluated with respect to manual segmentations of 11 structures for 3 subjects from the VISERAL challenge. State-of-the-art DIR methods based on Normalized Mutual Information (NMI), Modality Independent Neighborhood Descriptor (MIND) and their novel combination achieved a mean segmentation overlap ratio of 76.7, 67.7, 76.9%, respectively. This dropped to 69.1% or less when registering images synthesized by cycle-GAN based on local correlation, due to the poor performance on the thoracic region, where large lung volume changes were synthesized. Performance for the abdominal region was similar to that of CT-MRI NMI registration (77.4 vs. 78.8%) when using 3D synthesizing MRIs (12 slices) and medium sized receptive fields for the discriminator.

研究の動機と目的

• cycle-GANを用いて一方のモダリティからもう一方を合成する際の MR-CT の変形的画像登録 (DIR) を動機づけ、評価する。
• 合成画像の幾何学的ずれを緩和するかを検討するため、識別器の受容野を小さくすることの影響を調べる。
• 胸部および腹部ROIで、合成MR/CTを用いたDIRの性能を、強力なマルチモーダルDIRの基準（NMI、MIND、NMI+MIND）と比較して評価する。
• 領域サイズと合成パラメータが登録結果に与える影響を検討する。
• cycle-GANによる空間的バイアスを避けるためのモダリティデータセットのバランス取りについて指針を提供する。）
• method=[
• Adopt cycle-GANs to synthesize MR from CT and CT from MR using 2D residual-cycleGANs with PatchGAN discriminators.
• Train on 17 unpaired subjects per modality and evaluate 3D synthesis over thorax and abdomen ROIs.
• Incorporate cycle-consistency loss with a weighted objective that includes a cyclic L1 term (lambda_cyc = 10).
• Register CT/MR and synthesized images with a multi-modal DIR framework (ourDIR) that combines NMI and MIND dissimilarities.
• Use rigid initialization, then deformable registration with total variation or L2 regularization of the displacement field.
• Experiment with discriminator receptive field sizes (P×P patches) to study effects on geometric consistency.

提案手法

• cycle-GANを用いてCTからMRを、MRからCTを合成する 2D residual-cycleGANs を PatchGAN識別器とともに採用する。
• 各モダリティで17名の非ペア対象を用いて訓練し、胸部および腹部ROIsに対する3D合成を評価する。
• 循環整合性損失を、循環L1項を含む重み付き目的関数とともに組み込む（lambda_cyc = 10）。
• CT/MRおよび合成画像を、NMIと MIND の不類似性を組み合わせたマルチモーダルDIRフレームワーク（ourDIR）で登録する。
• 剛体初期化を用い、その後変位場の総変動正則化（total variation）またはL2正則化で変形登録を行う。
• 識別器の受容野サイズ（P×Pパッチ）を用いて、幾何学的一貫性への影響を調べる。）
• research_questions=["Can cycle-GAN synthesized MR/CT images enable accurate MR-CT deformable registration compared to standard multi-modal DIR measures?","How do region size and discriminator receptive field influence the geometric consistency and registration accuracy of synthesized images?","Do combinations of NMI and MIND dissimilarities improve DIR over single-modality or synthesis-based approaches?","What are the regional (thorax vs abdomen) differences in DIR performance when using synthesized images?"]
• research_questions_list:[]
• key_findings=[
• Registration with synthesized MR/CT can match or fall short of unpaired multi-modal methods depending on region; abdomen performance with synthesized data approaches CT-MR NMI results (~77.4–78.8% Dice range).
• Thoracic registrations show notable degradation with synthesized images due to lung-volume bias induced by cycle-GANs.
• Combining NMI and MIND (NMI+MIND) achieves competitive results for the thorax and abdomen, with best performance using the initial-gradient-based weighting (beta ≈ 0.8).
• 3D MR synthesis from CT and 3D CT synthesis from MR require careful region handling; inconsistent lung contours and slice-to-slice inconsistencies can occur, particularly with small ROI depths (C=3).
• Shallower discriminators (P=34) can reduce lung misalignment but may degrade synthesis quality, highlighting a trade-off between realism and geometric fidelity.

実験結果

リサーチクエスチョン

• RQ1 Can cycle-GAN synthesized MR/CT images enable accurate MR-CT deformable registration compared to standard multi-modal DIR measures?
• RQ2 How do region size and discriminator receptive field influence the geometric consistency and registration accuracy of synthesized images?
• RQ3 Do combinations of NMI and MIND dissimilarities improve DIR over single-modality or synthesis-based approaches?
• RQ4 What are the regional (thorax vs abdomen) differences in DIR performance when using synthesized images?

主な発見

• Registration with synthesized MR/CT can match or fall short of unpaired multi-modal methods depending on region; abdomen performance with synthesized data approaches CT-MR NMI results (~77.4–78.8% Dice range).
• Thoracic registrations show notable degradation with synthesized images due to lung-volume bias induced by cycle-GANs.
• Combining NMI and MIND (NMI+MIND) achieves competitive results for the thorax and abdomen, with best performance using the initial-gradient-based weighting (beta ≈ 0.8).
• 3D MR synthesis from CT and 3D CT synthesis from MR require careful region handling; inconsistent lung contours and slice-to-slice inconsistencies can occur, particularly with small ROI depths (C=3).
• Shallower discriminators (P=34) can reduce lung misalignment but may degrade synthesis quality, highlighting a trade-off between realism and geometric fidelity.