[论文解读] F3RNet: Full-Resolution Residual Registration Network for Multimodal Image Registration

Zhe Xu, Jie Luo+3 more|arXiv (Cornell University)|Sep 15, 2020

Medical Image Segmentation Techniques参考文献 17被引用 4

一句话总结

F3RNet 是一种新颖的无监督深度学习框架，用于多模态 CT-MRI 图像配准，通过并行流结合全分辨率特征学习与多尺度残差表示，实现亚3秒的推理时间，并在严重变形的腹部器官配准中表现出卓越的准确性，尤其在难以对齐的局部区域表现优异。

ABSTRACT

PURPOSE: Multimodal deformable image registration is essential for many image-guided therapies. Recently, deep learning approaches have gained substantial popularity and success in deformable image registration. Most deep learning approaches use the so-called mono-stream “high-to-low, low-to-high” network structure, and can achieve satisfactory overall registration results. However, accurate alignments for some severely deformed local regions, which are crucial for pinpointing surgical targets, are often overlooked, especially for multimodal inputs with vast intensity differences. Consequently, these approaches are not sensitive to some hard-to-align regions, e.g., intra-patient registration of deformed liver lobes. METHODS: We propose a novel unsupervised registration network, namely Full-Resolution Residual Registration Network (F3RNet), for multimodal registration of severely deformed organs. The proposed method combines two parallel processing streams in a residual learning fashion. One stream takes advantage of the full-resolution information that facilitates accurate voxel-level registration. The other stream learns the deep multi-scale residual representations to obtain robust recognition. We also factorize the 3D convolution to reduce the training parameters and enhance network efficiency. RESULTS: We validate the proposed method on 50 sets of clinically acquired intra-patient abdominal CT-MRI data. Experiments on both CT-to-MRI and MRI-to-CT registration demonstrate promising results compared to state-of-the-art approaches. CONCLUSION: By combining the high-resolution information and multi-scale representations in a highly interactive residual learning fashion, the proposed F3RNet can achieve accurate overall and local registration. The run time for registering a pair of CT-MRI images is less than 3 seconds using a GPU. In future works, we will investigate how to cost-effectively process high-resolution information and fuse multi-scale representations.

研究动机与目标

• 为解决多模态可变形图像配准中局部对齐不准确的挑战，特别是针对肝脏叶等严重变形器官。
• 提升对 CT 与 MRI 之间强度差异显著区域的对齐敏感性。
• 开发一种轻量化、高效的网络，在保持高分辨率空间细节的同时捕捉鲁棒的多尺度特征。
• 通过分解的 3D 卷积减少模型复杂度与训练参数，而不牺牲性能。

提出的方法

• F3RNet 采用双流架构：一条流保留全分辨率特征以实现精确的体素级对齐，另一条流学习深层多尺度残差表示以增强特征理解能力。
• 两条流通过残差学习进行融合，实现特征的交互式优化与表示学习的提升。
• 3D 卷积被分解为深度可分离卷积，以减少参数数量并提高训练效率。
• 网络采用无监督方式训练，使用变形后的移动图像与固定图像之间的相似性损失，无需真实变形场作为监督信号。

实验结果

研究问题

• RQ1 结合全分辨率与多尺度特征的双流网络架构是否能提升多模态 CT-MRI 图像中局部配准的准确性？
• RQ2 高分辨率流与多尺度流之间的残差学习在严重变形区域中如何增强配准性能？
• RQ3 分解的 3D 卷积在多大程度上降低了模型复杂度，同时保持了配准性能？
• RQ4 所提出的方法是否能在临床 3D 腹部扫描上实现近实时推理（每对图像配准时间低于 3 秒）？

主要发现

• F3RNet 在 50 组临床术前患者腹部 CT-MRI 数据中，实现了 CT 到 MRI 与 MRI 到 CT 配准的最先进性能。
• 该方法在严重变形区域（特别是肝脏叶等挑战性区域）表现出卓越的准确性。
• 使用 GPU 进行单对 CT-MRI 图像配准时，推理时间小于 3 秒，表明其具有极高的计算效率。
• 采用分解的 3D 卷积显著减少了训练参数数量，同时保持了高配准精度。