[Paper Review] Neural Radiance Fields in Medical Imaging: A Survey

Xin Wang, Chen, Yineng+4 more|arXiv (Cornell University)|Feb 26, 2024

Medical Imaging Techniques and Applications8 citations

TL;DR

This survey analyzes applying Neural Radiance Fields (NeRF) to medical imaging, outlines four core challenges, reviews organ-specific methods, datasets, and metrics, and suggests future research directions.

ABSTRACT

Neural Radiance Fields (NeRF), as a pioneering technique in computer vision, offer great potential to revolutionize medical imaging by synthesizing three-dimensional representations from the projected two-dimensional image data. However, they face unique challenges when applied to medical applications. This paper presents a comprehensive examination of applications of NeRFs in medical imaging, highlighting four imminent challenges, including fundamental imaging principles, inner structure requirement, object boundary definition, and color density significance. We discuss current methods on different organs and discuss related limitations. We also review several datasets and evaluation metrics and propose several promising directions for future research.

Motivation & Objective

• Identify the motivation for using NeRFs in medical imaging and the associated challenges.
• Categorize NeRF methodologies by organ type and discuss their limitations and data requirements.
• Review public datasets and evaluation metrics used in medical NeRF research.
• Outline future directions to improve detail, boundaries, color-density interpretation, and clinical integration.

Proposed method

• Classify NeRF applications by organ (knee, brain, heart vessels, skull, teeth, abdomen, chest) to highlight structural challenges.
• Discuss specific medical NeRF methods (e.g., MedNeRF, UMedNeRF, ACNeRF, mNeRF, SNAF, NAF) and their training/optimization strategies.
• Examine data generation with Digitally Reconstructed Radiographs (DRR) and the role of DRRs in simulating paired data.
• Review public datasets (MedNeRF, NAF, PerX2CT, DIF-Net) and evaluation metrics (PSNR, SSIM, FID, KID) used in medical NeRFs.
• Highlight limitations such as data quality, resolution, boundary ambiguity, modality generalization, and interpretability.
• Propose future directions including detail resolution, boundary delineation, computational efficiency, personalization, and integration with advanced techniques.

Experimental results

Research questions

• RQ1 What are the main imaging-principle and boundary-definition challenges when applying NeRFs to medical data?
• RQ2 How do NeRF-based methods perform across different anatomical organs and imaging modalities?
• RQ3 What public datasets and evaluation metrics are used to assess medical NeRFs, and what limitations do they reveal?
• RQ4 What future directions and integrations (e.g., with DRR, RL, foundation models) could advance NeRFs in clinical settings?

Key findings

• NeRFs offer detailed 3D representations from 2D medical images and can reduce radiation exposure and imaging time.
• Organ-specific NeRF methods face unique challenges like sparsity, overlap, and visibility of vessels, bones, and soft tissues.
• Several specialized models (MedNeRF, UMedNeRF, ACNeRF, mNeRF, SNAF, NAF) address alignment, uncertainty, and sparse-view reconstruction, yet hyperparameter balancing remains an issue.
• DRR-based synthetic X-rays enable paired-data training without extra patient exposure, supporting NeRF development.
• Public datasets for medical NeRFs are growing (e.g., MedNeRF, NAF, PerX2CT, DIF-Net), with metrics like PSNR, SSIM, FID, and KID used for evaluation.
• The survey outlines future directions including better detail, boundary delineation, color-density interpretation, and computational efficiency, plus potential integrations with 3D Gaussian splatting, reinforcement learning, and foundation models.