[Paper Review] Introduction to Camera Pose Estimation with Deep Learning

Yoli Shavit, Ron Ferens|arXiv (Cornell University)|Jul 8, 2019

Robotics and Sensor-Based Localization57 references51 citations

TL;DR

A survey of deep learning approaches for regressing absolute camera pose from RGB images, including method taxonomy, cross-comparisons, reproducibility notes, and future directions.

ABSTRACT

Over the last two decades, deep learning has transformed the field of computer vision. Deep convolutional networks were successfully applied to learn different vision tasks such as image classification, image segmentation, object detection and many more. By transferring the knowledge learned by deep models on large generic datasets, researchers were further able to create fine-tuned models for other more specific tasks. Recently this idea was applied for regressing the absolute camera pose from an RGB image. Although the resulting accuracy was sub-optimal, compared to classic feature-based solutions, this effort led to a surge of learning-based pose estimation methods. Here, we review deep learning approaches for camera pose estimation. We describe key methods in the field and identify trends aiming at improving the original deep pose regression solution. We further provide an extensive cross-comparison of existing learning-based pose estimators, together with practical notes on their execution for reproducibility purposes. Finally, we discuss emerging solutions and potential future research directions.

Motivation & Objective

• Motivate the study of learning-based camera pose estimation after early sub-optimal results.
• Summarize key deep learning approaches for regressing absolute camera pose from RGB imagery.
• Provide a cross-comparison of existing learning-based pose estimators.
• Offer practical notes to improve reproducibility and guide future research.

Proposed method

• Review and synthesize deep learning approaches for camera pose estimation.
• Describe key methods and identify trends improving pose regression performance.
• Provide an extensive cross-comparison of existing learning-based estimators.
• Discuss practical considerations for reproducibility and implementation.

Experimental results

Research questions

• RQ1 What are the main deep learning approaches used for camera pose estimation from RGB images?
• RQ2 How do different learning-based pose estimators compare in terms of performance, strengths, and weaknesses?
• RQ3 What practical considerations impact reproducibility and deployment of pose estimation methods?
• RQ4 What future directions and emerging solutions are identified for pose estimation with deep learning.

Key findings

• The paper reviews deep learning approaches for camera pose estimation and discusses trends toward improving the original regression-based solutions.
• It provides an extensive cross-comparison of existing learning-based pose estimators.
• It offers practical notes on executing methods to aid reproducibility.
• It discusses emerging solutions and potential future research directions.