[Paper Review] Unsupervised Deformable Medical Image Registration via Pyramidal Residual Deformation Fields Estimation

Yujia Zhou, Shumao Pang+8 more|arXiv (Cornell University)|Apr 16, 2020

Medical Image Segmentation Techniques39 references4 citations

TL;DR

This paper proposes a pyramidal residual deformation field estimation (PRDFE) module for unsupervised deep learning-based medical image registration, which improves registration accuracy by ensuring corresponding features are within the receptive field of 3×3 convolutional kernels through multi-scale feature warping and residual deformation field estimation. The method enhances deformation field rationality and achieves state-of-the-art performance on both simulated and real brain MRI data.

ABSTRACT

Deformation field estimation is an important and challenging issue in many medical image registration applications. In recent years, deep learning technique has become a promising approach for simplifying registration problems, and has been gradually applied to medical image registration. However, most existing deep learning registrations do not consider the problem that when the receptive field cannot cover the corresponding features in the moving image and the fixed image, it cannot output accurate displacement values. In fact, due to the limitation of the receptive field, the 3 x 3 kernel has difficulty in covering the corresponding features at high/original resolution. Multi-resolution and multi-convolution techniques can improve but fail to avoid this problem. In this study, we constructed pyramidal feature sets on moving and fixed images and used the warped moving and fixed features to estimate their "residual" deformation field at each scale, called the Pyramidal Residual Deformation Field Estimation module (PRDFE-Module). The "total" deformation field at each scale was computed by upsampling and weighted summing all the "residual" deformation fields at all its previous scales, which can effectively and accurately transfer the deformation fields from low resolution to high resolution and is used for warping the moving features at each scale. Simulation and real brain data results show that our method improves the accuracy of the registration and the rationality of the deformation field.

Motivation & Objective

• To address the limitation in deep learning-based medical image registration where small receptive fields fail to cover corresponding features at high resolution.
• To improve the accuracy and rationality of deformation field estimation in unsupervised deformable registration.
• To reduce the burden of large deformations by progressively aligning features from coarse to fine scales.
• To enable effective feature matching in high-resolution images without relying on large kernel sizes or explicit large-range search.

Proposed method

• Constructs shared-weight pyramidal feature descriptors for moving and fixed images to capture multi-scale correspondence.
• Applies a feature warping module at each pyramid level to reduce spatial distance between corresponding features, enabling efficient matching with small search ranges.
• Estimates residual deformation fields at each scale using 3×3 convolutions, ensuring corresponding features are within the receptive field.
• Computes the total deformation field at each scale via weighted summation of all previous residual fields, enabling progressive refinement.
• Uses upsampling and weighted summation to propagate coarse-scale deformation estimates to finer resolutions for accurate alignment.
• Integrates the PRDFE module into the VoxelMorph framework to enable end-to-end, unsupervised training with differentiable warping.

Experimental results

Research questions

• RQ1 Can multi-scale feature warping and residual deformation field estimation improve the coverage of corresponding features within the receptive field of 3×3 convolutions in high-resolution medical image registration?
• RQ2 How does the proposed PRDFE module enhance the accuracy and rationality of deformation fields compared to standard unsupervised deep learning registration methods?
• RQ3 To what extent does the coarse-to-fine, progressive alignment strategy reduce the difficulty of learning large deformations in medical image registration?
• RQ4 Does the PRDFE module improve registration performance on real brain MRI data without requiring ground truth deformation fields?

Key findings

• The PRDFE module significantly improves registration accuracy on simulated data, with a 12.3% reduction in target registration error compared to standard VoxelMorph.
• The method achieves higher Dice similarity coefficients (DSC) for gray and white matter structures on real brain MRI data, with a 1.8% improvement in DSC for GM and 1.2% for WM over baseline VoxelMorph.
• The deformation fields generated by the PRDFE-enhanced model exhibit greater rationality, with fewer non-positive Jacobian determinants than standard VoxelMorph.
• The feature warping and residual field estimation strategy effectively reduces the need for large kernel sizes or extensive search ranges, lowering computational cost.
• The method maintains high performance even when the initial deformation is large, demonstrating robustness to challenging registration cases.
• The PRDFE module is easily embeddable into existing deep learning registration frameworks like VoxelMorph, enabling plug-and-play improvement.