[Paper Review] Efficient Non-Line-of-Sight Imaging from Transient Sinograms

Mariko Isogawa, Dorian Chan+3 more|arXiv (Cornell University)|Aug 6, 2020

Advanced Optical Sensing Technologies33 references4 citations

TL;DR

This paper proposes a circular and confocal non-line-of-sight (C²NLOS) scanning method that reduces acquisition time and computational load by exploiting sinusoidal transient measurements—called a transient sinogram—enabling efficient 3D localization and 2D/3D image reconstruction from just 1.6% of the measurements used by prior state-of-the-art methods.

ABSTRACT

Non-line-of-sight (NLOS) imaging techniques use light that diffusely reflects off of visible surfaces (e.g., walls) to see around corners. One approach involves using pulsed lasers and ultrafast sensors to measure the travel time of multiply scattered light. Unlike existing NLOS techniques that generally require densely raster scanning points across the entirety of a relay wall, we explore a more efficient form of NLOS scanning that reduces both acquisition times and computational requirements. We propose a circular and confocal non-line-of-sight (C2NLOS) scan that involves illuminating and imaging a common point, and scanning this point in a circular path along a wall. We observe that (1) these C2NLOS measurements consist of a superposition of sinusoids, which we refer to as a transient sinogram, (2) there exists computationally efficient reconstruction procedures that transform these sinusoidal measurements into 3D positions of hidden scatterers or NLOS images of hidden objects, and (3) despite operating on an order of magnitude fewer measurements than previous approaches, these C2NLOS scans provide sufficient information about the hidden scene to solve these different NLOS imaging tasks. We show results from both simulated and real C2NLOS scans.

Motivation & Objective

• To reduce the number of measurements required for non-line-of-sight (NLOS) imaging while maintaining reconstruction quality.
• To identify a minimal measurement set that supports computationally efficient reconstruction of hidden scenes.
• To explore whether a circular scanning pattern on a relay wall can yield sufficient information for NLOS imaging tasks.
• To develop reconstruction algorithms that exploit the sinusoidal structure of transient measurements for speed and accuracy.
• To demonstrate that far fewer measurements than previously required can still enable high-quality 3D localization and image reconstruction.

Proposed method

• Proposes a circular and confocal scanning strategy where the laser source and sensor co-locate and scan a single point along a circular path on a visible relay wall.
• The transient measurements from C²NLOS form a superposition of sinusoids, termed a 'transient sinogram,' due to the geometric and temporal response of multiply scattered light.
• Employs a Hough voting procedure to localize discrete scatterers by estimating the frequency, amplitude, and phase of the sinusoidal components in the transient sinogram.
• Applies inverse Radon transform techniques to reconstruct 2D images of planar hidden objects by treating the transient sinogram as a projection of the hidden scene.
• Uses a linear least squares inversion for 2D imaging on large planar scenes, enabling fast and approximate reconstruction with minimal computational cost.
• Employs an ADMM-based optimization framework with strong priors for 3D volumetric reconstruction, enabling approximate 3D recovery from the transient sinogram.

Experimental results

Research questions

• RQ1 Can a circular scanning pattern on a relay wall produce sufficient transient measurements to reconstruct hidden scenes with fewer measurements than raster scanning?
• RQ2 Do the transient measurements from C²NLOS exhibit a sinusoidal structure that enables efficient reconstruction algorithms?
• RQ3 Can the parameters of the transient sinogram (frequency, amplitude, phase) be directly mapped to 3D positions of hidden scatterers?
• RQ4 Is it possible to reconstruct 2D and 3D images of hidden scenes using only a transient sinogram from C²NLOS, despite reduced measurement dimensionality?
• RQ5 How does the computational efficiency and reconstruction quality of C²NLOS compare to existing NLOS methods like LCT and FK?

Key findings

• C²NLOS requires only 1.6% of the capture time of LCT or FK methods while achieving similar localization accuracy, with mean estimation error within the same order of magnitude.
• The Hough voting procedure successfully localizes discrete scatterers using the sinusoidal parameters of the transient sinogram, enabling fast 3D position recovery.
• The inverse Radon transform applied to the transient sinogram enables 2D image reconstruction of planar scenes with visual quality comparable to LCT and FK, despite using far fewer measurements.
• Linear least squares inversion of C²NLOS measurements yields 2D image reconstructions that are slightly lower in SSIM than LCT or FK but are 50x faster to compute.
• 3D volumetric reconstructions using ADMM-based optimization with strong priors recover key structural features of hidden scenes (e.g., depth and pose), even though reconstruction quality is lower than LCT or FK.
• The transient sinogram from C²NLOS contains sufficient information to reconstruct approximate 3D appearances of non-planar objects, such as a bunny, by adjusting the effective focal depth via the scan radius.