[Paper Review] Simultaneous Localization And Mapping with depth Prediction using Capsule Networks for UAVs

Sunil Prakash, Gaelan Gu|arXiv (Cornell University)|Aug 16, 2018

Robotics and Sensor-Based Localization9 references4 citations

TL;DR

This paper proposes a monocular SLAM system for UAVs that leverages Capsule Networks (CapsNet) for depth prediction, improving upon CNN-based methods by encoding spatial hierarchies and object pose. The system integrates CapsNet-generated depth maps with keyframe-based camera pose estimation and an Extended Kalman Filter (EKF), achieving higher depth estimation accuracy than ORB-SLAM and CNN-SLAM on the TUM dataset, with 29.98% correct depth predictions on TUM/seq3.

ABSTRACT

In this paper, we propose an novel implementation of a simultaneous localization and mapping (SLAM) system based on a monocular camera from an unmanned aerial vehicle (UAV) using Depth prediction performed with Capsule Networks (CapsNet), which possess improvements over the drawbacks of the more widely-used Convolutional Neural Networks (CNN). An Extended Kalman Filter will assist in estimating the position of the UAV so that we are able to update the belief for the environment. Results will be evaluated on a benchmark dataset to portray the accuracy of our intended approach.

Motivation & Objective

• To develop a low-cost, monocular SLAM system for UAVs using depth prediction from Capsule Networks to overcome limitations of CNNs in encoding spatial pose and object orientation.
• To enhance depth estimation accuracy in single-camera UAV navigation by leveraging CapsNet's vector-based representation of spatial relationships.
• To integrate CapsNet-based depth maps with keyframe tracking and EKF-based pose estimation for robust 3D environment reconstruction.
• To evaluate the proposed method on benchmark datasets and compare its performance against ORB-SLAM and CNN-SLAM in depth prediction accuracy.
• To assess the generalization capability of the CapsNet model trained on TUM dataset when applied to a new, unseen indoor environment.

Proposed method

• The system uses a monocular forward-facing camera on a Parrot AR Quadcopter drone equipped with ROS, capturing 18fps, 640x480 video for SLAM processing.
• CapsNet processes input images to generate dense depth/disparity maps, encoding object pose and spatial hierarchy through dynamic routing between capsules.
• A keyframe-based approach detects and tracks 2D-3D correspondences, with camera pose estimated via EKF using optical flow and depth map constraints.
• The EKF fuses depth predictions and optical flow residuals, minimizing uncertainty using Huber norm and residual-based uncertainty modeling.
• An uncertainty map is generated by measuring pixel-wise confidence in predicted disparities to improve robustness in depth estimation.
• The system integrates depth maps with optical flow and keyframe data to build a global 3D map of the environment using EKF-based state estimation.

Experimental results

Research questions

• RQ1 Can Capsule Networks outperform Convolutional Neural Networks in depth prediction accuracy for monocular SLAM in UAV applications?
• RQ2 How does the vector-based representation in CapsNets improve spatial reasoning and object pose estimation compared to traditional CNNs?
• RQ3 To what extent does the CapsNet-based depth prediction enhance the accuracy of UAV pose estimation and 3D map reconstruction?
• RQ4 Does the model trained on the TUM dataset generalize well to new, unseen indoor environments with different lighting and structural features?
• RQ5 How does the integration of CapsNet depth maps with EKF and keyframe tracking improve overall SLAM performance compared to CNN-based or feature-based methods?

Key findings

• The proposed CapsNet-based SLAM system achieved 29.98% correct depth predictions on the TUM/seq3 dataset, outperforming both ORB-SLAM (12.477%) and CNN-SLAM (27.396%) in this sequence.
• On TUM/seq2, the method achieved 25.784% correct depth predictions, surpassing CNN-SLAM’s 24.077% and ORB-SLAM’s 0.059%.
• The system demonstrated generalization capability, achieving 28.590% correct depth predictions on a custom indoor apartment dataset, despite being trained on TUM data.
• The CapsNet-based depth prediction showed higher accuracy than CNN-SLAM in three out of four TUM sequences, indicating improved spatial reasoning and feature encoding.
• The uncertainty map generated from pixel-wise confidence improved robustness in depth estimation, particularly in textureless or low-contrast regions.
• The integration of CapsNet depth maps with EKF and optical flow enabled stable 3D mapping and accurate UAV pose estimation in real-time, despite monocular constraints.