[Paper Review] On the Continuity of Rotation Representations in Neural Networks

Yi Zhou, Connelly Barnes+3 more|arXiv (Cornell University)|Dec 17, 2018

3D Shape Modeling and AnalysisEngineering29 references57 citations

TL;DR

The paper defines a concept of continuity for rotation representations in neural networks, proves discontinuity for common 3D rotation representations in low dimensions, presents continuous high-dimensional representations (notably 5D and 6D) for SO(n), and empirically shows these continuous representations improve learning across autoencoding, rotation estimation, and inverse kinematics tasks.

ABSTRACT

In neural networks, it is often desirable to work with various representations of the same space. For example, 3D rotations can be represented with quaternions or Euler angles. In this paper, we advance a definition of a continuous representation, which can be helpful for training deep neural networks. We relate this to topological concepts such as homeomorphism and embedding. We then investigate what are continuous and discontinuous representations for 2D, 3D, and n-dimensional rotations. We demonstrate that for 3D rotations, all representations are discontinuous in the real Euclidean spaces of four or fewer dimensions. Thus, widely used representations such as quaternions and Euler angles are discontinuous and difficult for neural networks to learn. We show that the 3D rotations have continuous representations in 5D and 6D, which are more suitable for learning. We also present continuous representations for the general case of the n-dimensional rotation group SO(n). While our main focus is on rotations, we also show that our constructions apply to other groups such as the orthogonal group and similarity transforms. We finally present empirical results, which show that our continuous rotation representations outperform discontinuous ones for several practical problems in graphics and vision, including a simple autoencoder sanity test, a rotation estimator for 3D point clouds, and an inverse kinematics solver for 3D human poses.

Motivation & Objective

• Define a notion of continuity for representations used in neural networks and relate it to topology (homeomorphism and embedding).
• Characterize which rotation representations are continuous or discontinuous across 2D, 3D, and n-D rotations.
• Propose and formalize continuous representations for SO(n), including explicit constructions and dimensionality considerations.
• Demonstrate empirically that continuous representations outperform discontinuous ones in practical graphics/vision tasks.

Proposed method

• Introduce a formal definition of a continuous representation via a pair of mappings (f, g) with g continuous, and relate it to homeomorphism/embedding concepts.
• Analyze discontinuities of common 3D rotation representations (Euler angles, quaternions) and prove non-embeddability of SO(3) into low-dimensional Euclidean spaces (<5D).
• Construct continuous representations for SO(n) with dimensions n^2 - n (Case 3) and show how to reduce dimensionality while preserving continuity (Case 4 and projection-based methods).
• Specialize to 6D and 5D continuous representations for SO(3) via Gram-Schmidt-like embedding (Case 3) and stereographic projection reductions (Case 4).
• Discuss extension to O(n) and similarity transforms and provide network-implementation guidance (orthogonalization implications).
• Empirically compare continuous vs. discontinuous representations on rotation autoencoding, 3D point cloud rotation estimation, and 3D human pose inverse kinematics.

Experimental results

Research questions

• RQ1 Can rotation representations be made continuous for neural network training, and how does this impact learning efficiency and accuracy?
• RQ2 Which dimensions are required to embed SO(3) continuously into Euclidean space, and can continuous representations be constructed with practical dimensionality (5D/6D)?
• RQ3 Do continuous representations lead to measurable performance gains over traditional representations in autoencoding, pose estimation, and inverse kinematics tasks?
• RQ4 Can the proposed continuous representations be generalized to other groups such as O(n) and similarity transforms?

Key findings

• 3D rotation representations in low-dimensional Euclidean spaces (≤4D) are discontinuous, making learning difficult.
• Continuous representations for SO(n) exist in n^2 − n dimensions, with 5D and 6D cases demonstrated as practical for SO(3).
• A Gram-Schmidt-inspired 6D (and a 5D via projection) representation yields outputs that map back to valid rotations, preserving orthogonality.
• Empirical results show continuous representations outperform discontinuous ones across a rotation autoencoder, rotation estimation for 3D point clouds, and 3D human pose inverse kinematics, with markedly lower errors and faster convergence.
• Direct regression on 3x3 matrices yields larger errors than the proposed continuous representations.
• Orthogonalization within the network (or via postprocessing) interacts with representation choice, highlighting practical benefits of the higher-dimensional continuous forms.