[Paper Review] Rethinking Medical Image Reconstruction via Shape Prior, Going Deeper and Faster: Deep Joint Indirect Registration and Reconstruction

Jiulong Liu, Angelica I. Avilés-Rivero+2 more|arXiv (Cornell University)|Dec 16, 2019

Advanced MRI Techniques and Applications35 references4 citations

TL;DR

This paper proposes a novel deep joint framework that simultaneously performs indirect image registration and reconstruction from undersampled, noisy medical imaging data by integrating shape priors via learned diffeomorphic registration and physics-based inversion. By intertwining reconstruction and registration in a single differentiable model using deep residual blocks and fixed-point optimization, the method achieves state-of-the-art image quality with over 66% faster computation than prior methods across MRI, sparse-view CT, and low-dose CT.

ABSTRACT

Indirect image registration is a promising technique to improve image reconstruction quality by providing a shape prior for the reconstruction task. In this paper, we propose a novel hybrid method that seeks to reconstruct high quality images from few measurements whilst requiring low computational cost. With this purpose, our framework intertwines indirect registration and reconstruction tasks is a single functional. It is based on two major novelties. Firstly, we introduce a model based on deep nets to solve the indirect registration problem, in which the inversion and registration mappings are recurrently connected through a fixed-point interaction based sparse optimisation. Secondly, we introduce specific inversion blocks, that use the explicit physical forward operator, to map the acquired measurements to the image reconstruction. We also introduce registration blocks based deep nets to predict the registration parameters and warp transformation accurately and efficiently. We demonstrate, through extensive numerical and visual experiments, that our framework outperforms significantly classic reconstruction schemes and other bi-task method; this in terms of both image quality and computational time. Finally, we show generalisation capabilities of our approach by demonstrating their performance on fast Magnetic Resonance Imaging (MRI), sparse view computed tomography (CT) and low dose CT with measurements much below the Nyquist limit.

Motivation & Objective

• Address the challenge of reconstructing high-quality medical images from highly undersampled and noisy measurements, particularly in MRI with long acquisition times.
• Overcome the limitations of separate reconstruction and registration pipelines by jointly optimizing both tasks to reduce error propagation and improve generalization.
• Develop a computationally efficient method that maintains high image fidelity by leveraging deep learning for registration and reconstruction while respecting physical forward models.
• Demonstrate generalization across diverse imaging modalities, including fast MRI, sparse-view CT, and low-dose CT, with measurements far below the Nyquist limit.
• Introduce a hybrid model that unifies model-based regularization (LDDMM) with deep learning to enhance reconstruction accuracy and speed.

Proposed method

• Formulate a joint optimization problem that simultaneously solves indirect image registration and image reconstruction using a single functional, with shared parameters between the two tasks.
• Design deep residual networks with dedicated inversion blocks that explicitly incorporate the physical forward operators (e.g., Fourier, Radon transforms) to map raw measurements to image estimates.
• Implement registration blocks based on deep nets to predict deformation parameters and warping fields accurately and efficiently, enabling end-to-end differentiability.
• Use a fixed-point iterative scheme based on sparse optimization to couple the inversion and registration mappings through recurrent connections, ensuring stable and differentiable optimization.
• Train the entire network end-to-end using a differentiable loss function that combines data consistency, regularization via LDDMM, and reconstruction fidelity.
• Integrate shape priors from the registered template image into the reconstruction process to improve anatomical accuracy and reduce artifacts.

Experimental results

Research questions

• RQ1 Can indirect image registration, when jointly optimized with reconstruction, significantly improve image quality in undersampled medical imaging?
• RQ2 How can deep learning be effectively combined with model-based registration (LDDMM) to achieve both high reconstruction fidelity and low computational cost?
• RQ3 Can a unified deep learning framework outperform separate reconstruction and registration pipelines in terms of image quality and inference speed?
• RQ4 To what extent does the proposed method generalize across different imaging modalities (e.g., MRI, CT) and sampling patterns (e.g., Cartesian, radial)?
• RQ5 Can the integration of physical forward models within deep networks enhance reconstruction robustness and reduce artifacts in low-signal regimes?

Key findings

• The proposed method achieves the highest PSNR and SSIM scores among all compared methods on both MRI and CT datasets, with PSNR improvements significantly outperforming classic TV + LDDMM and prior state-of-the-art approaches.
• Reconstruction errors are consistently the lowest across all datasets and sampling patterns, indicating superior preservation of fine details, edges, and texture.
• The method reduces computational time by more than 66% compared to the classic TV + LDDMM scheme and the approach of Chen et al. [39], while maintaining or improving image quality.
• Visual results demonstrate that the proposed reconstructions preserve anatomical structures and avoid blurring and contrast loss, unlike baseline methods that introduce strong artifacts.
• The predicted momentum fields in Fig. 8 show accurate alignment with ground truth, confirming the effectiveness of the deep registration blocks in capturing complex deformations.
• Generalization is validated across multiple modalities—fast MRI, sparse-view CT, and low-dose CT—demonstrating robustness to low sampling rates and diverse acquisition patterns.