[Paper Review] Block-Coordinate Minimization for Large SDPs with Block-Diagonal Constraints

Yulun Tian, Kasra Khosoussi+1 more|arXiv (Cornell University)|Mar 2, 2019

Advanced Optimization Algorithms Research16 references4 citations

TL;DR

This paper extends block-coordinate minimization (BCM) with global convergence guarantees to optimization over the Cartesian product of Stiefel manifolds, enabling scalable solution of large-scale semidefinite programs (SDPs) with block-diagonal constraints. The method achieves a global convergence rate of $ O(1/k) $ for the Riemannian gradient norm, making it suitable for applications like rotation and pose synchronization.

ABSTRACT

The so-called Burer-Monteiro method is a well-studied technique for solving large-scale semidefinite programs (SDPs) via low-rank factorization. The main idea is to solve rank-restricted, albeit non-convex, surrogates instead of the SDP. Recent works have shown that, in an important class of SDPs with elegant geometric structure, one can find globally optimal solutions to the SDP by finding rank-deficient second-order critical points of an unconstrained Riemannian optimization problem. Hence, in such problems, the Burer-Monteiro approach can provide a scalable and reliable alternative to interior-point methods that scale poorly. Among various Riemannian optimization methods proposed, block-coordinate minimization (BCM) is of particular interest due to its simplicity. Erdogdu et al. in their recent work proposed BCM for problems over the Cartesian product of unit spheres and provided global convergence rate estimates for the algorithm. This report extends the BCM algorithm and the global convergence rate analysis of Erdogdu et al. from problems over the Cartesian product of unit spheres to the Cartesian product of Stiefel manifolds. The latter more general setting has important applications such as synchronization over the special orthogonal (SO) and special Euclidean (SE) groups.

Motivation & Objective

• Address the scalability limitations of interior-point methods for large-scale semidefinite programs (SDPs) with block-diagonal structure.
• Extend the block-coordinate minimization (BCM) algorithm and its global convergence analysis from unit spheres to the more general Stiefel manifold product $ \mathrm{St}(d,r)^n $.
• Enable efficient and reliable solution of large SDPs arising in synchronization problems over SO(d) and SE(d) groups.
• Provide theoretical convergence guarantees for BCM on Stiefel manifolds under both uniform and importance sampling schemes.
• Establish that rank-deficient second-order critical points of the Riemannian problem yield globally optimal solutions to the original SDP.

Proposed method

• Formulate the SDP as a low-rank optimization problem on the product Stiefel manifold $ \mathrm{St}(d,r)^n $, using the factorization $ X = Y^\top Y $.
• Decompose the cost function $ F(Y) = \operatorname{tr}(QY^\top Y) $ into block contributions $ F_i(Y_i) = \langle G_i, Y_i \rangle $, where $ G_i = \sum_{j \neq i} Y_j Q_{[j,i]} $.
• Apply block-coordinate minimization by iteratively optimizing one block $ Y_i $ while fixing others, using a closed-form solution via SVD: $ Y_i^\star = U_i I_{r \times d} V_i^\top $ for $ -G_i = U_i \Sigma_i V_i^\top $.
• Use two sampling strategies: uniform ($ p_i = 1/n $) and importance sampling ($ p_i \propto \|G_i\|_* $) to select blocks.
• Derive a global convergence rate bound of $ O(1/k) $ for the expected squared Riemannian gradient norm, showing $ \mathbb{E}\|\operatorname{grad}F(Y^k)\|_F^2 \leq \epsilon $ after $ O(1/\epsilon) $ iterations.
• Leverage matrix norm inequalities and eigenvalue/singular value interlacing to bound the decrease in objective per iteration.

Experimental results

Research questions

• RQ1 Can block-coordinate minimization be generalized from the product of unit spheres to the product of Stiefel manifolds for large-scale SDPs?
• RQ2 What global convergence rate can be established for BCM on $ \mathrm{St}(d,r)^n $ under uniform and importance sampling?
• RQ3 Does the Riemannian BCM algorithm converge to a globally optimal solution when the solution is rank-deficient and second-order critical?
• RQ4 How does the convergence rate depend on problem parameters such as $ d $, $ n $, and the nuclear norm of $ Q $?
• RQ5 Can the algorithm be applied effectively to synchronization problems over SO(d) and SE(d) with large-scale instances?

Key findings

• The proposed BCM algorithm on $ \mathrm{St}(d,r)^n $ achieves a global convergence rate of $ O(1/k) $ for the expected squared Riemannian gradient norm.
• The convergence bound depends on the problem size via $ dnC_1(Q) $, where $ C_1(Q) = \sum_{i \neq j} \|Q_{[i,j]}\|_* $, and the initial objective gap.
• Importance sampling improves convergence by prioritizing blocks with higher gradient contribution, leading to faster convergence in practice.
• The method guarantees global optimality when a rank-deficient second-order critical point is found, as such points correspond to globally optimal solutions of the original SDP.
• Theoretical analysis confirms that the algorithm avoids saddle points and converges to a critical point with high probability under mild assumptions.
• The approach is scalable to large problems, such as pose-graph optimization with $ d \times n > 10^4 $, where interior-point methods are infeasible.