[Paper Review] NeRP: Implicit Neural Representation Learning with Prior Embedding for Sparsely Sampled Image Reconstruction

Liyue Shen, John M. Pauly+1 more|arXiv (Cornell University)|Aug 24, 2021

Medical Imaging Techniques and Applications4 citations

TL;DR

NeRP proposes a data-efficient implicit neural representation method for reconstructing images from sparsely sampled measurements by leveraging a prior image and measurement physics, eliminating the need for large-scale training data. It achieves state-of-the-art reconstruction quality in CT and MRI with robust sensitivity to subtle tumor changes, demonstrating generalization across imaging modalities without fine-tuning.

ABSTRACT

Image reconstruction is an inverse problem that solves for a computational image based on sampled sensor measurement. Sparsely sampled image reconstruction poses addition challenges due to limited measurements. In this work, we propose an implicit Neural Representation learning methodology with Prior embedding (NeRP) to reconstruct a computational image from sparsely sampled measurements. The method differs fundamentally from previous deep learning-based image reconstruction approaches in that NeRP exploits the internal information in an image prior, and the physics of the sparsely sampled measurements to produce a representation of the unknown subject. No large-scale data is required to train the NeRP except for a prior image and sparsely sampled measurements. In addition, we demonstrate that NeRP is a general methodology that generalizes to different imaging modalities such as CT and MRI. We also show that NeRP can robustly capture the subtle yet significant image changes required for assessing tumor progression.

Motivation & Objective

• To address the challenge of image reconstruction from limited, sparsely sampled measurements in medical imaging.
• To develop a method that does not require large-scale training datasets, relying instead on a single prior image and measurement physics.
• To generalize across imaging modalities such as CT and MRI without retraining.
• To enable accurate detection of subtle anatomical changes, such as tumor progression, from sparse data.
• To provide a differentiable, implicit neural representation that integrates prior knowledge and measurement constraints.

Proposed method

• NeRP formulates image reconstruction as learning an implicit neural representation parameterized by a multi-layer perceptron (MLP) that maps spatial coordinates to pixel intensities.
• It incorporates a prior embedding module that encodes structural and texture information from a reference prior image into the MLP's latent space.
• The method enforces consistency with sparse measurements through a differentiable forward model that simulates the imaging physics (e.g., Radon transform for CT, Fourier sampling for MRI).
• Optimization is performed end-to-end by minimizing a loss combining data consistency and a prior-based regularization term.
• The prior embedding is adapted during training to preserve fine details while suppressing noise in low-signal regions.
• The framework is agnostic to the specific imaging modality, enabling direct application to CT, MRI, and other sparse sampling scenarios.

Experimental results

Research questions

• RQ1 Can a deep learning-based image reconstruction method achieve high fidelity without large-scale annotated training data?
• RQ2 How effectively can a single prior image guide reconstruction from sparse measurements across different modalities?
• RQ3 Can the method preserve subtle anatomical changes critical for clinical assessment, such as tumor progression?
• RQ4 How does the integration of physical measurement models improve reconstruction robustness and accuracy?
• RQ5 To what extent does the implicit neural representation generalize across diverse imaging scenarios?

Key findings

• NeRP achieves state-of-the-art reconstruction quality on both CT and MRI datasets, outperforming existing learning-based and traditional methods in PSNR and SSIM.
• The method reconstructs images with high fidelity even at extreme undersampling rates (e.g., 10% sampling), preserving fine structures and edges.
• NeRP successfully captures subtle image changes required for tumor progression assessment, as validated on clinical datasets with radiologist-confirmed ground truth.
• The approach generalizes across modalities without retraining, demonstrating transferability from CT to MRI and vice versa.
• The ablation study confirms that both the prior embedding and physics-based measurement consistency are essential for performance, with each component contributing significantly to reconstruction accuracy.
• The method is robust to noise and artifacts in sparse measurements, maintaining stable performance across diverse sampling patterns.