[Paper Review] An Analysis of SVD for Deep Rotation Estimation

Jake Levinson, Carlos Esteves+5 more|arXiv (Cornell University)|Jun 25, 2020

Robotics and Sensor-Based LocalizationEngineering42 references32 citations

TL;DR

The paper argues that projecting neural network outputs onto SO(3) via SVDO+ (SVD-based orthogonalization) is theoretically optimal and empirically state-of-the-art for 3D rotation estimation in diverse supervised and unsupervised settings.

ABSTRACT

Symmetric orthogonalization via SVD, and closely related procedures, are well-known techniques for projecting matrices onto $O(n)$ or $SO(n)$. These tools have long been used for applications in computer vision, for example optimal 3D alignment problems solved by orthogonal Procrustes, rotation averaging, or Essential matrix decomposition. Despite its utility in different settings, SVD orthogonalization as a procedure for producing rotation matrices is typically overlooked in deep learning models, where the preferences tend toward classic representations like unit quaternions, Euler angles, and axis-angle, or more recently-introduced methods. Despite the importance of 3D rotations in computer vision and robotics, a single universally effective representation is still missing. Here, we explore the viability of SVD orthogonalization for 3D rotations in neural networks. We present a theoretical analysis that shows SVD is the natural choice for projecting onto the rotation group. Our extensive quantitative analysis shows simply replacing existing representations with the SVD orthogonalization procedure obtains state of the art performance in many deep learning applications covering both supervised and unsupervised training.

Motivation & Objective

• Motivate the question of which rotation representation is best for deep learning models estimating 3D rotations.
• Propose SVDO+ (SVD-based orthogonalization to SO(3)) as a natural and optimal projection method.
• Provide theoretical analysis and empirical evidence across multiple tasks (point cloud alignment, object pose from images, inverse kinematics, depth prediction).
• Compare SVDO+ to Gram-Schmidt and other rotation representations in both supervised and unsupervised settings.

Proposed method

• Define SVDO(M)=UV^T and SVDO+(M)=UΣ′V^T as projections onto O(n) and SO(n) respectively.
• Argue SVDO+ minimizes the Frobenius norm to M among SO(n) projections (MLE under Gaussian noise).
• Derive gradients for SVDO/ SVDO+ layers to show stability and favorable backpropagation properties.
• Provide error analysis showing SVDO+ has half the expected reconstruction error of Gram-Schmidt under Gaussian noise (n=3).
• Benchmark against 6D/5D representations and classic parameterizations (quaternion, Euler, axis-angle) across tasks.

Experimental results

Research questions

• RQ1 Is SVDO+ the natural and optimal projection for 3D rotations in neural networks?
• RQ2 How does SVDO+ compare to Gram-Schmidt and other rotation representations in reconstruction error and training stability?
• RQ3 Do SVDO+ approaches achieve state-of-the-art performance across supervised and unsupervised rotation estimation tasks?
• RQ4 What are the theoretical and empirical behaviors of SVDO+ gradients in training?
• RQ5 How does SVDO+ perform on diverse 3D rotation tasks like point-cloud alignment, pose estimation from images, inverse kinematics, and depth prediction?

Key findings

• SVDO+ minimizes the reconstruction error to M over SO(n) and is optimal under Gaussian noise (MLE).
• Error analysis shows SVDO produces half the expected error of Gram-Schmidt for small Gaussian perturbations in 3D.
• SVDO-based methods achieve state-of-the-art or competitive performance across four application areas and both supervised and unsupervised settings.
• SVDO+ remains smooth and differentiable except in degenerate det(M)<0 or zero-small-singular-value scenarios, but is generally well-behaved for 3D.
• SVD-Train (training with SVDO+) often outperforms alternatives, including 6D and many classic representations, in the reported benchmarks.