[Paper Review] Learned holographic light transport

Koray Kavaklı, Hakan Ürey+1 more|arXiv (Cornell University)|Aug 1, 2021

Advanced Optical Imaging Technologies21 references4 citations

TL;DR

This paper proposes a learned holographic light transport model that improves simulation accuracy in holographic displays by learning a single complex-valued convolution kernel from real-world camera captures of holographic reconstructions. By training on phase-only holograms optimized via ideal simulations, the method achieves significantly higher image quality and realism in simulations, outperforming standard ideal models without complex architectures or extensive calibration.

ABSTRACT

Computer-Generated Holography (CGH) algorithms often fall short in matching simulations with results from a physical holographic display. Our work addresses this mismatch by learning the holographic light transport in holographic displays. Using a camera and a holographic display, we capture the image reconstructions of optimized holograms that rely on ideal simulations to generate a dataset. Inspired by the ideal simulations, we learn a complex-valued convolution kernel that can propagate given holograms to captured photographs in our dataset. Our method can dramatically improve simulation accuracy and image quality in holographic displays while paving the way for physically informed learning approaches.

Motivation & Objective

• Address the persistent gap between simulated and physical holographic display performance.
• Reduce reliance on complex deep learning models or extensive experimental calibration for holographic display modeling.
• Develop a simple, differentiable method to learn the true light transport of a physical holographic display.
• Enable more accurate simulations that reflect real-world image quality for phase-only holograms.
• Provide a foundation for physically informed learning in holography with minimal parameter overhead.

Proposed method

• Capture real image reconstructions from a proof-of-concept phase-only holographic display using a camera.
• Optimize input holograms using an ideal, fully differentiable holographic light transport model (Rayleigh-Sommerfeld diffraction).
• Train a single complex-valued convolutional kernel to map ideal holograms to captured physical reconstructions.
• Use a differentiable loss function comparing simulated reconstructions (via learned kernel) to actual camera images.
• Learn a point-spread function that models the aggregate optical aberrations and imperfections of the physical display.
• Apply the learned kernel in simulation to propagate holograms to the image plane with improved realism.

Experimental results

Research questions

• RQ1 Can a single learned complex convolutional kernel effectively model the non-ideal light transport of a real holographic display?
• RQ2 To what extent can this learned kernel improve simulation accuracy compared to idealized holographic models?
• RQ3 Does this method reduce the need for complex deep learning architectures or extensive calibration procedures?
• RQ4 How does the learned kernel compare in performance and parameter efficiency to state-of-the-art neural network-based approaches?
• RQ5 Can the learned kernel capture and preserve key image quality metrics such as dynamic range and brightness?

Key findings

• The learned holographic light transport method significantly improves simulation accuracy, achieving image quality that closely matches real-world physical reconstructions.
• The method preserves dynamic range and brightness levels better than the ideal simulation model, as evidenced by visual comparisons in Figure 4.
• The approach reduces the number of tunable parameters to four million (2×1080×1920 for amplitude and phase), halving the parameter count compared to state-of-the-art methods using deep neural networks.
• The learned kernel acts as a point-spread function that captures the aggregate optical imperfections of the display, enabling more realistic simulations.
• The method avoids the locality issue of small kernel sizes by using a kernel size equal to the input image, ensuring full spatial context in propagation.
• The approach enables physically informed learning with minimal experimental and algorithmic overhead, offering a simple yet effective alternative to complex GANs or multi-layer models.