[Paper Review] Single-shot ToF sensing with sub-mm precision using conventional CMOS sensors

Manuel Ballester, Heming Wang+3 more|arXiv (Cornell University)|Dec 2, 2022

Advanced Optical Sensing Technologies4 citations

TL;DR

This paper presents a single-shot time-of-flight (ToF) camera using synthetic wavelength interferometry with conventional CMOS sensors to achieve sub-millimeter depth precision. By illuminating objects with multiple wavelengths and exploiting interferometric phase unwrapping, the system captures full 3D models in one exposure, enabling motion-robust, high-resolution 3D imaging of dynamic and rough-surfaced objects with up to 330 µm depth precision.

ABSTRACT

We present a novel single-shot interferometric ToF camera targeted for precise 3D measurements of dynamic objects. The camera concept is based on Synthetic Wavelength Interferometry, a technique that allows retrieval of depth maps of objects with optically rough surfaces at submillimeter depth precision. In contrast to conventional ToF cameras, our device uses only off-the-shelf CCD/CMOS detectors and works at their native chip resolution (as of today, theoretically up to 20 Mp and beyond). Moreover, we can obtain a full 3D model of the object in single-shot, meaning that no temporal sequence of exposures or temporal illumination modulation (such as amplitude or frequency modulation) is necessary, which makes our camera robust against object motion. In this paper, we introduce the novel camera concept and show first measurements that demonstrate the capabilities of our system. We present 3D measurements of small (cm-sized) objects with > 2 Mp point cloud resolution (the resolution of our used detector) and up to sub-mm depth precision. We also report a "single-shot 3D video" acquisition and a first single-shot "Non-Line-of-Sight" measurement. Our technique has great potential for high-precision applications with dynamic object movement, e.g., in AR/VR, industrial inspection, medical imaging, and imaging through scattering media like fog or human tissue.

Motivation & Objective

• To develop a motion-robust, high-precision 3D imaging system for dynamic and optically rough surfaces.
• To overcome the limitations of conventional ToF cameras, which suffer from low depth precision and require temporal modulation or multiple exposures.
• To enable single-shot 3D reconstruction using only off-the-shelf CMOS sensors without specialized sensor architectures.
• To extend synthetic wavelength interferometry to dynamic and macroscopic rough-surfaced objects in a single-shot framework.
• To demonstrate feasibility for non-line-of-sight (NLoS) imaging using a single-shot interferometric ToF approach.

Proposed method

• The system uses multiple laser wavelengths (λ₁, λ₂, etc.) to create synthetic wavelengths (Λ) that resolve ambiguous phase unwrapping in interferometric measurements.
• Interference patterns between object and reference beams are captured in a single exposure using a conventional CMOS sensor at native pixel resolution (up to 20 Mp).
• Phase maps are extracted from the Fourier domain of the captured image, and synthetic wavelength interferometry is applied to disambiguate the phase and retrieve depth.
• The method leverages multi-wavelength interferometry to resolve random phase fluctuations caused by speckle on rough surfaces, enabling depth precision beyond the limits of single-wavelength interferometry.
• A computational pipeline reconstructs full 3D point clouds from the unwrapped phase maps, with lateral resolution limited only by sensor resolution and aperture design.
• The system is designed to operate with minimal optical path differences between reference and object beams, ensuring stable interference and high-precision depth retrieval.

Experimental results

Research questions

• RQ1 Can synthetic wavelength interferometry be adapted to achieve sub-millimeter depth precision in a single-shot ToF camera using only standard CMOS sensors?
• RQ2 Can the system maintain high depth precision and motion robustness when imaging dynamic or optically rough surfaces?
• RQ3 Is it feasible to extend the single-shot interferometric ToF approach to non-line-of-sight (NLoS) imaging scenarios?
• RQ4 How does the system's performance scale with multiple laser wavelengths and synthetic wavelength design?
• RQ5 What are the fundamental trade-offs between aperture size, field of view, and spectral separation in the Fourier domain?

Key findings

• The system achieves a depth precision of up to 330 µm, representing a 100× improvement over conventional ToF cameras.
• The camera successfully captures a single-shot 3D video of a moving object, demonstrating robustness to motion without temporal modulation.
• The system enables the first single-shot non-line-of-sight (NLoS) measurement using synthetic wavelength interferometry, extending the technique beyond line-of-sight imaging.
• Full 3D point clouds with resolution exceeding 2 Mp were reconstructed from a single exposure, limited only by the sensor’s native resolution.
• The method is extendable to three or more wavelengths, enabling multiple synthetic wavelengths and improved phase unwrapping in a single-shot configuration.
• The system operates within the constraints of standard CMOS sensors and does not require specialized sensor architectures or temporal multiplexing.