[Paper Review] Probabilistic orientation estimation with matrix Fisher distributions

David Mohlin, G. Bianchi+1 more|arXiv (Cornell University)|Jun 17, 2020

Robotics and Sensor-Based LocalizationEngineering31 references18 citations

TL;DR

This paper proposes a deep learning method for probabilistic 3D orientation estimation using matrix Fisher distributions to model rotation uncertainty. By training a neural network to predict the unconstrained parameters of this distribution and optimizing via a convex negative log-likelihood loss, the approach achieves state-of-the-art performance on Pascal3D+, ModelNet10-SO(3), and UPNA head pose datasets, with improved training stability and uncertainty calibration.

ABSTRACT

This paper focuses on estimating probability distributions over the set of 3D rotations ($SO(3)$) using deep neural networks. Learning to regress models to the set of rotations is inherently difficult due to differences in topology between $\mathbb{R}^N$ and $SO(3)$. We overcome this issue by using a neural network to output the parameters for a matrix Fisher distribution since these parameters are homeomorphic to $\mathbb{R}^9$. By using a negative log likelihood loss for this distribution we get a loss which is convex with respect to the network outputs. By optimizing this loss we improve state-of-the-art on several challenging applicable datasets, namely Pascal3D+, ModelNet10-$SO(3)$ and UPNA head pose.

Motivation & Objective

• To address the challenge of estimating probability distributions over SO(3), the manifold of 3D rotations, using deep neural networks.
• To overcome the topological mismatch between the unconstrained outputs of neural networks (R^9) and the nonlinear, closed manifold SO(3).
• To enable stable and effective training for 3D rotation estimation by using a loss function that is convex and has bounded gradients.
• To provide a probabilistic framework that models both the mode and uncertainty of 3D rotations without requiring complex constraint enforcement.
• To demonstrate superior performance and robustness on multiple benchmark datasets compared to existing methods.

Proposed method

• The method uses a deep neural network to regress the parameters of a matrix Fisher distribution, which are unconstrained and homeomorphic to R^9, enabling direct regression.
• The negative log-likelihood of the matrix Fisher distribution is used as the training loss, which is convex with respect to the network outputs, ensuring stable optimization.
• A differentiable approximation is used to efficiently compute the intractable normalizing constant of the matrix Fisher distribution during training.
• The network is trained end-to-end to predict the full distribution over rotations, allowing for both mode estimation and uncertainty quantification.
• The method includes data augmentation, class embedding, and image pre-processing via homography to improve generalization and robustness.
• The predicted matrix Fisher distribution is used to estimate the most likely rotation and its associated uncertainty for downstream tasks.

Experimental results

Research questions

• RQ1 Can a deep neural network be trained to output a full probability distribution over 3D rotations using an unconstrained parameterization?
• RQ2 Does using the negative log-likelihood of the matrix Fisher distribution as a loss lead to a convex optimization problem with bounded gradients?
• RQ3 Can this approach achieve state-of-the-art performance on 3D rotation estimation benchmarks such as Pascal3D+, ModelNet10-SO(3), and UPNA head pose?
• RQ4 How does the method handle rotational symmetries and ambiguous poses, and what is its behavior in uncertainty estimation?
• RQ5 What is the relative contribution of data augmentation, class embedding, and image pre-processing to the overall performance?

Key findings

• The proposed method achieves state-of-the-art performance on the Pascal3D+ dataset, with a median error of 8.9 degrees and accuracy of 90.8% at π/6.
• On the ModelNet10-SO(3) benchmark, the method outperforms previous approaches, demonstrating strong generalization to object-level 3D rotation estimation.
• On the UPNA head pose dataset, the method achieves high accuracy, showing robustness to real-world head pose variations.
• Ablation studies show that data augmentation is the most critical component, while class embedding and homography-based pre-processing provide marginal or no additional benefit on Pascal3D+.
• The method exhibits stable training due to the convexity of the negative log-likelihood loss, avoiding the local minima issues common in Euler angle or quaternion-based approaches.
• The network learns to model uncertainty appropriately: early in training, ambiguous axes are assigned nearly uniform distributions, and as training progresses, uncertainty decreases when the correct pose is learned.