[Paper Review] Zero-Shot Learning of Continuous 3D Refractive Index Maps from Discrete Intensity-Only Measurements

Renhao Liu, Yu Sun+3 more|arXiv (Cornell University)|Nov 27, 2021

Digital Holography and Microscopy60 references4 citations

TL;DR

DeCAF is the first neural field-based method for zero-shot 3D refractive index (RI) reconstruction from limited-angle, intensity-only measurements in intensity diffraction tomography (IDT). It learns a continuous, artifact-free RI volume representation directly from data without external training, achieving high-contrast reconstructions across multiple IDT modalities and biological samples.

ABSTRACT

Intensity diffraction tomography (IDT) refers to a class of optical microscopy techniques for imaging the 3D refractive index (RI) distribution of a sample from a set of 2D intensity-only measurements. The reconstruction of artifact-free RI maps is a fundamental challenge in IDT due to the loss of phase information and the missing cone problem. Neural fields (NF) has recently emerged as a new deep learning (DL) paradigm for learning continuous representations of complex 3D scenes without external training datasets. We present DeCAF as the first NF-based IDT method that can learn a high-quality continuous representation of a RI volume directly from its intensity-only and limited-angle measurements. We show on three different IDT modalities and multiple biological samples that DeCAF can generate high-contrast and artifact-free RI maps.

Motivation & Objective

• Address the fundamental challenge of reconstructing artifact-free 3D refractive index (RI) maps from intensity-only, limited-angle measurements in intensity diffraction tomography (IDT).
• Overcome the missing cone problem and phase loss inherent in intensity-only measurements by learning a continuous 3D RI representation.
• Enable zero-shot learning of RI volumes without requiring external annotated training datasets, leveraging neural fields for implicit scene representation.
• Develop a method that generalizes across diverse IDT modalities and biological samples while maintaining high reconstruction fidelity and contrast.

Proposed method

• Utilize neural fields (NF) to implicitly represent the 3D refractive index volume as a continuous, differentiable function of spatial coordinates.
• Train the neural field using only intensity-only projections acquired at limited angles, without requiring phase or ground-truth RI data.
• Formulate the forward model as a differentiable simulation of light propagation through the sample, enabling backpropagation through the imaging process.
• Optimize the neural field parameters by minimizing the difference between predicted and measured intensities using a differentiable imaging simulator.
• Leverage the inductive bias of neural fields to implicitly regularize the solution and suppress artifacts, even with incomplete data.
• Enable zero-shot inference by directly predicting continuous RI maps from raw intensity measurements without fine-tuning on specific samples.

Experimental results

Research questions

• RQ1 Can a neural field-based approach reconstruct high-quality, continuous 3D refractive index maps from intensity-only, limited-angle measurements without external training data?
• RQ2 How well does the proposed method generalize across different IDT modalities and diverse biological samples?
• RQ3 To what extent does the method mitigate artifacts and the missing cone problem compared to conventional IDT reconstruction techniques?
• RQ4 Can the implicit regularization of neural fields enable artifact-free reconstructions even with severely incomplete data?

Key findings

• DeCAF successfully reconstructs high-contrast, artifact-free 3D refractive index maps from intensity-only, limited-angle measurements without requiring any external training data.
• The method generalizes across three distinct IDT modalities and multiple biological samples, demonstrating robustness and zero-shot capability.
• By leveraging the inductive bias of neural fields, DeCAF effectively mitigates the missing cone problem and suppresses reconstruction artifacts.
• The continuous representation learned by DeCAF enables high-fidelity RI volume reconstruction directly from raw intensity projections, outperforming conventional methods in visual quality and structural fidelity.