[Paper Review] Multiple Scattering Media Imaging via End-to-End Neural Network

Ziyang Yuan, Hongxia Wang|arXiv (Cornell University)|Jun 20, 2018

Random lasers and scattering media14 references3 citations

TL;DR

This paper proposes TCNN, an end-to-end deep neural network that directly reconstructs images from phaseless speckle patterns in multiple scattering media without requiring transmission matrix estimation. By integrating a transform layer to bridge domains and learning the inverse mapping end-to-end, TCNN achieves state-of-the-art image recovery quality with over 100× faster inference than conventional phase retrieval methods, reducing per-image recovery time to under 0.12 seconds for 64×64 images.

ABSTRACT

Recovering the image of an object from its phaseless speckle pattern is difficult. Let alone the transmission matrix is unknown in multiple scattering media imaging. Double phase retrieval is a recently proposed efficient method which recovers the unknown object from its phaseless measurements by two steps with phase retrieval. In this paper, we combine the two steps in double phase retrieval and construct an end-to-end neural network called TCNN(Transforming Convolutional Neural Network) which directly learns the relationship between the phaseless measurements and the object. TCNN contains a special layer called transform layer which aims to be a bridge between different transform domains. Tested by the empirical data provided in\cite{Metzler2017Coherent}, images can be recovered by TCNN with comparable quality compared with state-of-the-art methods. Not only the transmission matrix needn't to be calculated but also the time to recover the object can be hugely reduced once the parameters of TCNN are stable.

Motivation & Objective

• To address the challenge of image recovery from phaseless speckle patterns in multiple scattering media, where traditional methods require costly transmission matrix estimation.
• To overcome the ill-posed nature of phase retrieval in complex scattering environments with unknown wavefront distortions.
• To develop an end-to-end deep learning framework that directly maps intensity measurements to object images, eliminating intermediate steps.
• To significantly reduce image recovery time compared to existing phase retrieval algorithms while maintaining competitive reconstruction quality.
• To validate the effectiveness of a novel transform layer that enables domain transfer within a convolutional neural network for inverse imaging.

Proposed method

• TCNN is a deep neural network that learns the inverse mapping from phaseless speckle intensity patterns to object images in a single end-to-end training process.
• The network architecture consists of a nonlinear encoding-decoding block followed by a linear transformation layer, with a specialized 'transform layer' that facilitates cross-domain representation learning.
• The transform layer acts as a learnable bridge between different transform domains (e.g., Fourier and spatial), enabling efficient feature transformation critical for inverse imaging.
• The network is trained using paired data of original objects and their corresponding measured speckle intensities, with loss minimized via mean squared error (MSE) on the reconstruction.
• Training is performed once offline using large-scale data, after which inference is highly efficient and suitable for real-time applications.
• Regularization techniques such as early stopping and validation-based monitoring are used to prevent overfitting, with training convergence observed within 20 epochs.

Experimental results

Research questions

• RQ1 Can an end-to-end deep neural network effectively reconstruct images from phaseless speckle patterns in multiple scattering media without requiring transmission matrix estimation?
• RQ2 How does the performance of the proposed TCNN compare to state-of-the-art phase retrieval methods in terms of image quality and inference speed?
• RQ3 What is the impact of the novel transform layer in enabling domain transfer within the network for improved image reconstruction?
• RQ4 To what extent can TCNN reduce recovery time compared to iterative phase retrieval algorithms like Wirtinger Flow or Fienup’s method?
• RQ5 Can TCNN generalize across different image sizes and scattering conditions while maintaining high reconstruction fidelity?

Key findings

• TCNN achieves image reconstruction quality comparable to state-of-the-art methods such as PRVAMP, GS, WF, and PhaseLift, particularly preserving object outlines despite loss of fine details.
• For 64×64 images, TCNN reduces per-image recovery time to 0.121 seconds, representing a 100× speedup compared to the slowest baseline (PhaseLift at 239.5 seconds).
• The training error and validation error converge rapidly, with minimal gap after 20 epochs, indicating effective control of overfitting.
• The network maintains low error even on small validation sets, demonstrating robustness and generalization capability across different image sizes.
• The time cost of TCNN is consistently below 0.12 seconds per image across all tested sizes, significantly outperforming iterative methods that require 10–100 iterations.
• The end-to-end design eliminates the need for separate transmission matrix estimation, streamlining the imaging pipeline and enabling real-time deployment.