[Paper Review] Leveraging Planar Regularities for Point Line Visual-Inertial Odometry

Xin Li, Yijia He+2 more|arXiv (Cornell University)|Apr 16, 2020

Robotics and Sensor-Based Localization26 references4 citations

TL;DR

This paper proposes PLP-VIO, a tightly-coupled monocular visual-inertial odometry system that integrates point features, line features, and planar regularities to improve 3D pose estimation and dense mesh reconstruction. By leveraging co-planarity constraints and a non-iterative plane detection method, the system achieves higher localization accuracy and more robust, structure-aware dense mapping than state-of-the-art methods, especially in texture-poor environments with abundant lines.

ABSTRACT

With monocular Visual-Inertial Odometry (VIO) system, 3D point cloud and camera motion can be estimated simultaneously. Because pure sparse 3D points provide a structureless representation of the environment, generating 3D mesh from sparse points can further model the environment topology and produce dense mapping. To improve the accuracy of 3D mesh generation and localization, we propose a tightly-coupled monocular VIO system, PLP-VIO, which exploits point features and line features as well as plane regularities. The co-planarity constraints are used to leverage additional structure information for the more accurate estimation of 3D points and spatial lines in state estimator. To detect plane and 3D mesh robustly, we combine both the line features with point features in the detection method. The effectiveness of the proposed method is verified on both synthetic data and public datasets and is compared with other state-of-the-art algorithms.

Motivation & Objective

• To address the limitations of sparse point-based monocular VIO systems that lack structural information and produce inaccurate or incomplete dense maps.
• To improve localization accuracy in textureless or low-texture environments where point features are sparse or unreliable.
• To develop a tightly-coupled VIO system that jointly optimizes pose, 3D landmarks, and planar structures using heterogeneous visual features.
• To enable efficient, real-time dense mesh generation from sparse 3D points and spatial lines by exploiting planar regularities.
• To validate the effectiveness of combining point, line, and plane features in improving both localization and mapping performance.

Proposed method

• The system extends PL-VIO by incorporating plane detection and 3D mesh generation using both 3D point and 3D line features as input.
• A non-iterative plane detection method is proposed that uses co-planarity constraints derived from detected 3D lines and points to improve robustness and speed.
• 2D Delaunay triangulation is applied to 2D point and line features to generate an initial 2D mesh, which is then projected into 3D to form a 3D mesh using 3D landmarks.
• Co-planarity constraints are enforced during sliding window optimization to improve the accuracy of 3D point and line estimation.
• The system uses the Schur complement trick to accelerate optimization and marginalization, maintaining real-time performance despite increased state vector size.
• Plane and mesh generation are performed in the back end after 3D triangulation of 2D features, with plane detection based on geometric consistency of 3D points and lines.

Experimental results

Research questions

• RQ1 Can the integration of line features and planar regularities significantly improve localization accuracy in monocular VIO, especially in texture-poor environments?
• RQ2 How does combining point, line, and plane features affect the quality and completeness of dense 3D mesh reconstruction compared to point-only methods?
• RQ3 Can a non-iterative plane detection method achieve comparable or better accuracy than RANSAC-based approaches while being significantly faster?
• RQ4 To what extent do co-planarity constraints enhance the accuracy of 3D landmark estimation in a tightly-coupled VIO framework?
• RQ5 Does the use of structural lines improve the robustness and correctness of plane detection and mesh generation when point features are sparse?

Key findings

• On the EuRoC MH_05 dataset, PLP-VIO achieved the lowest translation error in RPE evaluation, particularly outperforming other methods in the 57–67s range where point features were sparse.
• The mean mapping error was reduced to 2.8 cm with PLP-VIO compared to 3.6 cm with the point-only method (P), demonstrating improved reconstruction accuracy.
• The proposed non-iterative plane detection method achieved a mean execution time of 0.44 ms, significantly faster than RANSAC-based alternatives, and maintained high accuracy.
• Mesh creation with PLP-VIO took only 1.42 ms on average, and the total optimization time remained under 36 ms per frame, enabling real-time performance.
• In scenes with few point features but abundant structural lines (e.g., Fig. 8), PLP-VIO maintained high accuracy, while the P and PP methods degraded, proving the value of line and plane features.
• Ablation studies confirmed that the combination of points, lines, and planes (PLP) consistently outperformed individual components (P, PL, PP), with PLP showing the most robust and accurate results.