[Paper Review] InfiniTAM v3: A Framework for Large-Scale 3D Reconstruction with Loop Closure

Victor Adrian Prisacariu, Olaf Kähler+5 more|arXiv (Cornell University)|Aug 2, 2017

Robotics and Sensor-Based Localization7 references86 citations

TL;DR

InfiniTAM v3 presents a scalable 3D reconstruction framework using hash-based voxel hashing, robust tracking, a random ferns relocaliser, and globally consistent submap optimization for loop closure.

ABSTRACT

Volumetric models have become a popular representation for 3D scenes in recent years. One breakthrough leading to their popularity was KinectFusion, which focuses on 3D reconstruction using RGB-D sensors. However, monocular SLAM has since also been tackled with very similar approaches. Representing the reconstruction volumetrically as a TSDF leads to most of the simplicity and efficiency that can be achieved with GPU implementations of these systems. However, this representation is memory-intensive and limits applicability to small-scale reconstructions. Several avenues have been explored to overcome this. With the aim of summarizing them and providing for a fast, flexible 3D reconstruction pipeline, we propose a new, unifying framework called InfiniTAM. The idea is that steps like camera tracking, scene representation and integration of new data can easily be replaced and adapted to the user's needs. This report describes the technical implementation details of InfiniTAM v3, the third version of our InfiniTAM system. We have added various new features, as well as making numerous enhancements to the low-level code that significantly improve our camera tracking performance. The new features that we expect to be of most interest are (i) a robust camera tracking module; (ii) an implementation of Glocker et al.'s keyframe-based random ferns camera relocaliser; (iii) a novel approach to globally-consistent TSDF-based reconstruction, based on dividing the scene into rigid submaps and optimising the relative poses between them; and (iv) an implementation of Keller et al.'s surfel-based reconstruction approach.

Motivation & Objective

• Advance KinectFusion-style 3D reconstruction to large-scale environments via a hash-based volumetric representation.
• Improve camera tracking robustness and include relocalisation for tracking failures.
• Enable globally consistent reconstruction through submaps and inter-submap constraints.
• Provide surfel-based reconstruction as an optional alternative within the same framework.

Proposed method

• Use a hash table to map voxel blocks to a global voxel block array enabling scalable 3D representation.
• Embed a cross-device engine architecture with abstract, device-specific, and device-agnostic layers for modularity and performance.
• Implement six-stage per-frame processing: tracking, allocation, swapping in, integration, raycasting, and swapping out.
• Enhance tracking with ITMDepthTracker, ITMColorTracker, and ITMExtendedTracker, including RGB-D and photometric terms.
• Incorporate a random ferns relocaliser for camera pose relocalisation and loop-closure detection.
• Offer globally-consistent reconstruction via submaps and inter-submap constraint optimization (multi-engine pipeline).

Experimental results

Research questions

• RQ1 How can InfiniTAM scale KinectFusion-style reconstruction to large scenes using voxel hashing?
• RQ2 How can tracking be made robust and capable of relocalisation when tracking fails?
• RQ3 How can loop closure be detected and global consistency achieved in large-scale reconstructions?
• RQ4 What are the trade-offs between TSDF-based and surfel-based representations within a single framework?
• RQ5 How can cross-device architectures be organized to optimize performance across CPUs, GPUs, and other accelerators?

Key findings

• A robust tracking module is developed and integrated with a relocaliser to recover poses after tracking failures.
• A globally-consistent reconstruction approach based on submaps and inter-submap constraints is introduced.
• Surfel-based reconstruction is implemented as an optional alternative to TSDFs within InfiniTAM v3.
• Hash-table based voxel hashing scales to large scenes while keeping the active map in device memory via swapping.
• A random ferns relocaliser is implemented to enable relocalisation and loop closure detection for global consistency.