[Paper Review] Learning Personalized Representation for Inverse Problems in Medical Imaging Using Deep Neural Network

Kuang Gong, Kyung Sang Kim+5 more|arXiv (Cornell University)|Jul 4, 2018

Medical Imaging Techniques and Applications21 references3 citations

TL;DR

This paper proposes a personalized deep learning framework for medical image inverse problems that uses only a patient’s own prior images—no paired training data—by embedding a deep neural network as a data-driven regularizer within an ADMM optimization. The method outperforms conventional techniques in brain PET image reconstruction and denoising, achieving superior contrast-to-noise ratio and structural accuracy without requiring high-dose or long-scan reference images.

ABSTRACT

Recently deep neural networks have been widely and successfully applied in computer vision tasks and attracted growing interests in medical imaging. One barrier for the application of deep neural networks to medical imaging is the need of large amounts of prior training pairs, which is not always feasible in clinical practice. In this work we propose a personalized representation learning framework where no prior training pairs are needed, but only the patient's own prior images. The representation is expressed using a deep neural network with the patient's prior images as network input. We then applied this novel image representation to inverse problems in medical imaging in which the original inverse problem was formulated as a constraint optimization problem and solved using the alternating direction method of multipliers (ADMM) algorithm. Anatomically guided brain positron emission tomography (PET) image reconstruction and image denoising were employed as examples to demonstrate the effectiveness of the proposed framework. Quantification results based on simulation and real datasets show that the proposed personalized representation framework outperform other widely adopted methods.

Motivation & Objective

• To address the challenge of limited availability of high-quality training pairs in deep learning for medical imaging, especially when high-dose or long-scan acquisitions pose safety or practical constraints.
• To develop a personalized representation learning framework that leverages only the patient’s own prior images as input, eliminating the need for external labeled datasets.
• To integrate deep neural networks as adaptive regularizers within a constrained optimization framework for improved image quality in inverse problems such as PET reconstruction and denoising.
• To demonstrate the effectiveness of the framework in clinical brain PET imaging applications, showing improved quantitative and visual performance over state-of-the-art methods.

Proposed method

• The framework employs an alternating direction method of multipliers (ADMM) to decouple the inverse problem into a penalized optimization step and a deep network training step.
• A modified 3D U-Net with approximately 1.46 million trainable parameters is used as the neural network architecture to learn personalized image representations from the patient’s prior images.
• The network is trained end-to-end using the patient’s own prior images as input, with no need for paired ground-truth labels, making it suitable for low-dose or limited-scan clinical scenarios.
• The ADMM framework ensures data consistency by enforcing constraints during optimization, while the deep network acts as a learned regularizer that encodes patient-specific anatomical structures.
• The method is applied to two clinical tasks: anatomically guided brain PET reconstruction and whole-body PET denoising, using CT images as prior inputs.
• The optimization process alternates between solving a data-fidelity subproblem and updating the network parameters via backpropagation, enabling joint learning and reconstruction.

Experimental results

Research questions

• RQ1 Can a deep neural network learn a personalized image representation from a patient’s own prior images without requiring paired training data?
• RQ2 How does the integration of a deep network as a learned regularizer improve image reconstruction and denoising in low-dose or noisy PET imaging?
• RQ3 Does the proposed ADMM-based framework maintain data consistency while enabling effective end-to-end training of the network?
• RQ4 How does the performance of the proposed method compare quantitatively to conventional methods such as Gaussian filtering, non-local means, and kernel-based approaches in PET image restoration?
• RQ5 Can the framework generalize across different clinical scenarios, including brain and whole-body PET imaging, with consistent improvements in image quality?

Key findings

• The proposed method achieved a mean CNR of 49.29 across 10 patient datasets in whole-body PET denoising, significantly outperforming the Gaussian method (41.11) and NLM method (45.18).
• In brain PET reconstruction, the proposed method showed clearer cortical structures and reduced white matter noise compared to kernel-based methods, with improved lesion contrast and reduced artifacts.
• The CR-STD curve demonstrated that the proposed method achieved the best trade-off between contrast and standard deviation, indicating superior image quality and noise suppression.
• The method outperformed state-of-the-art techniques in both qualitative and quantitative evaluations, particularly in preserving small structures and enhancing lesion visibility.
• The framework showed robust performance on real clinical datasets, with consistent improvements across multiple patient studies and imaging protocols.
• Despite the lack of theoretical convergence guarantees, the method exhibited monotonic convergence in practice, as observed in the optimization trajectory.