[Paper Review] Stochastic Primal-Dual Method on Riemannian Manifolds with Bounded Sectional Curvature

Masoud Badiei Khuzani, Na Li|arXiv (Cornell University)|Mar 23, 2017

Stochastic Gradient Optimization Techniques45 references4 citations

TL;DR

This paper proposes a stochastic primal-dual algorithm for constrained optimization on Riemannian manifolds with bounded sectional curvature, leveraging geodesic convexity and comparison geometry. It establishes non-asymptotic convergence rates that explicitly depend on the manifold’s sectional curvature, with tight bounds for hyperbolic, elliptic, and asymptotically elliptic manifolds.

ABSTRACT

We study a stochastic primal-dual method for constrained optimization over Riemannian manifolds with bounded sectional curvature. We prove non-asymptotic convergence to the optimal objective value. More precisely, for the class of hyperbolic manifolds, we establish a convergence rate that is related to the sectional curvature lower bound. To prove a convergence rate in terms of sectional curvature for the elliptic manifolds, we leverage Toponogov's comparison theorem. In addition, we provide convergence analysis for the asymptotically elliptic manifolds, where the sectional curvature at each given point on manifold is locally bounded from below by the distance function. We demonstrate the performance of the primal-dual algorithm on the sphere for the non-negative principle component analysis (PCA). In particular, under the non-negativity constraint on the principle component and for the symmetric spiked covariance model, we empirically show that the primal-dual approach outperforms the spectral method. We also examine the performance of the primal-dual method for the anchored synchronization from partial noisy measurements of relative rotations on the Lie group SO(3). Lastly, we show that the primal-dual algorithm can be applied to the weighted MAX-CUT problem under constraints on the admissible cut. Specifically, we propose different approximation algorithms for the weighted MAX-CUT problem based on optimizing a function on the manifold of direct products of the unit spheres as well as the manifold of direct products of the rotation groups.

Motivation & Objective

• To develop a scalable, projection-free optimization method for constrained problems on Riemannian manifolds where the feasible set is non-convex in Euclidean space.
• To establish non-asymptotic convergence guarantees for stochastic primal-dual methods on manifolds with bounded sectional curvature.
• To unify convergence analysis across different curvature regimes—hyperbolic (negative), elliptic (positive), and asymptotically elliptic (locally bounded below by distance function).
• To demonstrate practical efficacy on non-negative PCA, anchored synchronization on SO(3), and constrained weighted MAX-CUT problems.

Proposed method

• Uses a Riemannian primal-dual algorithm that operates directly on the manifold, avoiding Euclidean embeddings and projection steps.
• Applies stochastic subgradient methods with implicit regularization via the manifold’s geometry, ensuring primal iterates remain on the manifold.
• For hyperbolic manifolds, derives convergence rates dependent on the lower bound of sectional curvature using geodesic convexity and curvature-aware analysis.
• For elliptic manifolds, leverages Toponogov’s comparison theorem to bound curvature effects and derive convergence rates involving sectional curvature.
• Extends analysis to asymptotically elliptic manifolds by bounding sectional curvature from below using the squared distance function from a fixed point.
• Employs Orlicz norm concentration inequalities and Talagrand-type bounds to control stochastic error in the subgradient estimates.

Experimental results

Research questions

• RQ1 Can a stochastic primal-dual method achieve non-asymptotic convergence on Riemannian manifolds with bounded sectional curvature?
• RQ2 How does the sectional curvature influence the convergence rate in hyperbolic and elliptic manifolds?
• RQ3 Can the convergence analysis be extended to manifolds where sectional curvature is bounded below by a function of distance rather than a constant?
• RQ4 Does the Riemannian primal-dual method outperform standard spectral methods in non-negative PCA under low signal-to-noise ratios?
• RQ5 Can the method be effectively applied to synchronization problems on Lie groups like SO(3) and constrained MAX-CUT on product manifolds?

Key findings

• For hyperbolic manifolds with bounded sectional curvature, the algorithm achieves a global non-asymptotic convergence rate that depends explicitly on the curvature lower bound.
• For elliptic manifolds, Toponogov’s comparison theorem enables a convergence bound that incorporates the sectional curvature, with the bound approaching the hyperbolic case as curvature tends to zero.
• The convergence analysis is extended to asymptotically elliptic manifolds, where the sectional curvature is bounded below by a quadratic function of the distance from a fixed point.
• In non-negative PCA under a symmetric spiked covariance model, the primal-dual method outperforms the spectral method in low SNR regimes under cosine similarity, particularly due to the non-negativity constraint.
• The method scales gracefully with sample size, outperforming semi-definite programming (SDP) approaches that are limited to sample sizes in the hundreds.
• Empirical results show the algorithm effectively solves anchored synchronization on SO(3) from partial noisy relative rotation measurements, demonstrating robustness and scalability.