[Paper Review] Spherical Regression: Learning Viewpoints, Surface Normals and 3D Rotations on n-Spheres

Shuai Liao, Efstratios Gavves+1 more|arXiv (Cornell University)|Apr 10, 2019

Robotics and Sensor-Based Localization34 references4 citations

TL;DR

This paper proposes spherical regression for learning 3D orientations by modeling viewpoints, surface normals, and 3D rotations on n-spheres using exponential normalization. By replacing standard normalization with spherical normalization via the exponential function, the method improves generalization and alignment with ground truth, achieving state-of-the-art performance on Pascal3D+ and ModelNet10-SO3 benchmarks across viewpoint estimation, surface normal prediction, and 3D rotation regression tasks.

ABSTRACT

Many computer vision challenges require continuous outputs, but tend to be solved by discrete classification. The reason is classification's natural containment within a probability $n$-simplex, as defined by the popular softmax activation function. Regular regression lacks such a closed geometry, leading to unstable training and convergence to suboptimal local minima. Starting from this insight we revisit regression in convolutional neural networks. We observe many continuous output problems in computer vision are naturally contained in closed geometrical manifolds, like the Euler angles in viewpoint estimation or the normals in surface normal estimation. A natural framework for posing such continuous output problems are $n$-spheres, which are naturally closed geometric manifolds defined in the $\mathbb{R}^{(n+1)}$ space. By introducing a spherical exponential mapping on $n$-spheres at the regression output, we obtain well-behaved gradients, leading to stable training. We show how our spherical regression can be utilized for several computer vision challenges, specifically viewpoint estimation, surface normal estimation and 3D rotation estimation. For all these problems our experiments demonstrate the benefit of spherical regression. All paper resources are available at https://github.com/leoshine/Spherical_Regression.

Motivation & Objective

• Address the challenge of learning continuous 3D orientations (viewpoints, normals, rotations) with deep neural networks.
• Overcome limitations of standard ℓ₂ normalization in regression tasks on spherical manifolds.
• Improve generalization and alignment with ground truth by modeling predictions on n-spheres using exponential normalization.
• Demonstrate consistent performance gains across multiple 3D orientation prediction benchmarks: Pascal3D+, NYUv2, and ModelNet10-SO3.
• Provide theoretical derivation of the Jacobian for spherical normalization to support training stability and gradient flow.

Proposed method

• Propose a spherical normalization module, $ S_{ ext{exp}}^n $, replacing standard ℓ₂ normalization to map outputs onto the n-sphere.
• Use exponential activation $ f(o_i) = e^{o_i} $ in the normalization layer to enforce sparsity and directional preference in predictions.
• Derive the analytical Jacobian of the spherical normalization layer as $ oldsymbol{J}_{ ext{exp}} = (oldsymbol{I} - oldsymbol{P} oldsymbol{P}^T) ext{diag}(oldsymbol{P}) $, enabling stable backpropagation.
• Integrate the spherical normalization module as a plug-in head after a backbone network (e.g., ResNet101 or VGG16) for end-to-end training.
• Apply the method to three tasks: viewpoint estimation (Euler angles), surface normal prediction (on NYUv2), and 3D rotation regression (quaternions on ModelNet10-SO3).
• Use a two-head prediction head: one for absolute value (via softmax on discrete bins) and one for sign prediction, improving localization accuracy.

Experimental results

Research questions

• RQ1 Can spherical normalization improve the generalization and alignment of deep neural networks in 3D orientation prediction tasks compared to standard ℓ₂ normalization?
• RQ2 How does the proposed $ S_{ ext{exp}}^n $ normalization perform on viewpoint estimation with fine-grained angular metrics on Pascal3D+?
• RQ3 Does spherical regression preserve sharper boundaries in surface normal prediction while maintaining detail, compared to smooth regression baselines?
• RQ4 To what extent does spherical normalization improve 3D rotation estimation accuracy, especially on symmetric objects?
• RQ5 What is the theoretical impact of spherical normalization on gradient flow, as captured by the Jacobian of the normalization layer?

Key findings

• On Pascal3D+, the proposed $ S_{ ext{exp}}^1 $ module improves Acc@π/6 to 88.2% (from 83.6% in Penedones et al.), with a MedErr of 9.2° (vs. 11.6°), showing consistent gains across all categories.
• For surface normal estimation on NYUv2, the spherical regression method produces sharper boundaries than Zhang et al. [9], preserving more geometric details despite slightly less smoothness.
• On ModelNet10-SO3, $ S_{ ext{exp}}^3 $ improves 3D rotation estimation accuracy over flat VGG16 regression, especially on symmetric objects like bathtub and table, with higher Acc@π/6.
• The method achieves a class-wise Acc@π/6 of 93% on car and 98% on airplane, outperforming prior methods on challenging categories.
• Theoretical analysis confirms that $ oldsymbol{J}_{ ext{exp}} = (oldsymbol{I} - oldsymbol{P} oldsymbol{P}^T) ext{diag}(oldsymbol{P}) $ provides better gradient behavior than $ oldsymbol{J}_{ ext{flat}} $, supporting improved training stability.
• The spherical regression module consistently improves performance across all evaluated tasks and benchmarks, demonstrating broad applicability to 3D geometric learning.