[Paper Review] Visual-Inertial Navigation: A Concise Review

Guoquan Huang|arXiv (Cornell University)|Jun 6, 2019

Robotics and Sensor-Based Localization164 references18 citations

TL;DR

This paper provides a concise yet comprehensive review of visual-inertial navigation systems (VINS), focusing on state estimation, sensor calibration, and observability analysis. It surveys key VINS algorithms—including EKF, MSCKF, and optimization-based methods—highlighting their strengths, limitations, and recent advances such as preintegration and observability-aware design, with the goal of accelerating research and development in the field.

ABSTRACT

As inertial and visual sensors are becoming ubiquitous, visual-inertial navigation systems (VINS) have prevailed in a wide range of applications from mobile augmented reality to aerial navigation to autonomous driving, in part because of the complementary sensing capabilities and the decreasing costs and size of the sensors. In this paper, we survey thoroughly the research efforts taken in this field and strive to provide a concise but complete review of the related work -- which is unfortunately missing in the literature while being greatly demanded by researchers and engineers -- in the hope to accelerate the VINS research and beyond in our society as a whole.

Motivation & Objective

• To address the lack of a contemporary, focused literature review on visual-inertial navigation systems (VINS), which is critical for researchers and engineers.
• To provide a concise but complete overview of VINS research, emphasizing core aspects such as state estimation, sensor calibration, and observability analysis.
• To clarify the strengths and limitations of major VINS approaches, including EKF, MSCKF, and optimization-based methods, to guide algorithm selection and development.
• To identify and discuss open challenges in VINS, such as persistent localization, semantic mapping, and integration with alternative sensors.
• To accelerate progress in VINS by synthesizing state-of-the-art knowledge and fostering deeper understanding of fundamental principles like observability and bias estimation.

Proposed method

• The paper employs a systematic survey methodology, organizing VINS research around core components: state estimation, sensor calibration, and observability analysis.
• It reviews the extended Kalman filter (EKF) framework, detailing the state vector that includes IMU states (orientation, velocity, position, biases) and feature positions.
• The IMU kinematic model is described using continuous-time dynamics with quaternion-based rotation propagation and bias drift models driven by zero-mean Gaussian noise.
• The paper explains the camera measurement model, including projection of 3D feature points into 2D image coordinates via the pinhole camera model.
• It analyzes observability properties of VINS, identifying that standard EKF suffers from a three-dimensional unobservable subspace instead of the expected four, leading to filter inconsistency.
• Techniques such as the first-estimates Jacobian (FEJ) and observability-constrained (OC) formulations are discussed to improve filter consistency and estimation accuracy.

Experimental results

Research questions

• RQ1 What are the fundamental challenges in designing consistent and accurate visual-inertial state estimators, particularly regarding observability and bias estimation?
• RQ2 How do different VINS algorithm families—EKF, MSCKF, and optimization-based methods—compare in terms of accuracy, efficiency, and robustness?
• RQ3 Why does the standard EKF fail to maintain proper observability in VINS, and what techniques can restore it?
• RQ4 What are the key open challenges in enabling long-term, large-scale, and safety-critical VINS deployments?
• RQ5 How can VINS be extended to incorporate semantic understanding, dynamic object tracking, and alternative sensors like LiDAR or event cameras?

Key findings

• The standard EKF-based VINS suffers from observability issues due to a three-dimensional unobservable subspace, leading to inconsistent state estimates despite accurate measurements.
• Techniques such as the first-estimates Jacobian (FEJ) and observability-constrained (OC) formulations significantly improve filter consistency by aligning the linearization with the system's true unobservable subspace.
• Preintegration of IMU measurements enables efficient integration of high-rate inertial data into graph-based optimization frameworks, improving computational efficiency and accuracy.
• While EKF-based methods remain popular due to their efficiency, optimization-based approaches (e.g., windowed optimization) have become dominant in state-of-the-art systems due to better scalability and robustness.
• Current VINS systems are not yet robust for long-term, large-scale deployments due to resource constraints and drift accumulation, especially in dynamic or low-texture environments.
• Future directions such as semantic mapping, high-dimensional object tracking, and cooperative VINS show strong potential but remain underexplored, indicating significant research opportunities.