[Paper Review] Recent Advances in Imaging Around Corners

Tomohiro Maeda, Guy Satat+3 more|arXiv (Cornell University)|Oct 12, 2019

Advanced Optical Sensing TechnologiesPhysics and Astronomy88 references47 citations

TL;DR

This is a comprehensive survey of non-line-of-sight (NLOS) imaging using light, reviewing ToF, coherence, and intensity-based techniques, hardware, algorithms, and real-world challenges.

ABSTRACT

Seeing around corners, also known as non-line-of-sight (NLOS) imaging is a computational method to resolve or recover objects hidden around corners. Recent advances in imaging around corners have gained significant interest. This paper reviews different types of existing NLOS imaging techniques and discusses the challenges that need to be addressed, especially for their applications outside of a constrained laboratory environment. Our goal is to introduce this topic to broader research communities as well as provide insights that would lead to further developments in this research area.

Motivation & Objective

• Introduce non-line-of-sight imaging to a broad research audience.
• Review existing NLOS imaging techniques and sensing modalities.
• Discuss forward models, reconstruction and inference algorithms, and their applicability to real-world use.
• Highlight hardware options and practical challenges for deployment beyond the lab.

Proposed method

• Classify NLOS techniques into Time-of-Flight, coherence-based, and intensity-based categories.
• Describe forward models mapping hidden scenes to measurements and discuss ill-posedness and regularization.
• Summarize reconstruction algorithms (back-projection, optimization with priors, confocal deconvolution, wave-based and inverse rendering approaches).
• Explain inference strategies for localization, tracking, and classification versus full scene reconstruction.
• Outline hardware choices (streak cameras, SPADs, AMCW ToF cameras, traditional cameras, interferometers) and their trade-offs.
• Discuss data-driven approaches and the role of neural networks in inference and recognition tasks.

Experimental results

Research questions

• RQ1 What are the existing techniques to see around corners and how do they differ by sensing modality?
• RQ2 What forward models and reconstruction/inference algorithms enable NLOS imaging across ToF, coherence, and intensity approaches?
• RQ3 How do hardware choices and noise/illumination conditions affect practical NLOS imaging outside laboratory settings?
• RQ4 What are the main challenges and potential solutions for real-time, robust NLOS imaging in real-world deployments?
• RQ5 How can data-driven methods contribute to localization, tracking, and classification around corners?

Key findings

• Time-of-Flight methods provide ellipsoidal constraints on hidden object locations by leveraging three-bounce photon paths.
• Confocal imaging reduces the forward model to a 3D convolution, enabling faster deconvolution-based reconstruction.
• Coherence-based methods exploit memory effects and spatial coherence to reconstruct or infer hidden scenes despite scattering.
• Intensity-based methods rely on occlusions or wall reflectance, enabling real-time passive tracking and reconstruction with priors.
• Multiple hardware modalities (streak cameras, SPADs, AMCW ToF, traditional cameras, interferometers) offer different trade-offs in time resolution, SNR, cost, and practicality.
• Data-driven approaches show promise for localization and classification, though generalization remains an active area of research.