[Paper Review] The Projected Power Method: An Efficient Algorithm for Joint Alignment from Pairwise Differences

Yuxin Chen, Emmanuel J. Candès|arXiv (Cornell University)|Sep 19, 2016

Sparse and Compressive Sensing Techniques32 references19 citations

TL;DR

This paper proposes the Projected Power Method, an efficient algorithm for joint alignment from noisy pairwise modulo differences, by lifting discrete variables into a continuous space and using projected power iterations to optimize over hypercubes. It proves the method achieves exact recovery (maximum likelihood estimate) under broad statistical models with high probability when sample complexity is sufficient.

ABSTRACT

Various applications involve assigning discrete label values to a collection of objects based on some pairwise noisy data. Due to the discrete---and hence nonconvex---structure of the problem, computing the optimal assignment (e.g.~maximum likelihood assignment) becomes intractable at first sight. This paper makes progress towards efficient computation by focusing on a concrete joint alignment problem---that is, the problem of recovering $n$ discrete variables $x_i \in \{1,\cdots, m\}$, $1\leq i\leq n$ given noisy observations of their modulo differences $\{x_i - x_j~\mathsf{mod}~m\}$. We propose a low-complexity and model-free procedure, which operates in a lifted space by representing distinct label values in orthogonal directions, and which attempts to optimize quadratic functions over hypercubes. Starting with a first guess computed via a spectral method, the algorithm successively refines the iterates via projected power iterations. We prove that for a broad class of statistical models, the proposed projected power method makes no error---and hence converges to the maximum likelihood estimate---in a suitable regime. Numerical experiments have been carried out on both synthetic and real data to demonstrate the practicality of our algorithm. We expect this algorithmic framework to be effective for a broad range of discrete assignment problems.

Motivation & Objective

• To address the computational intractability of discrete assignment problems with nonconvex, combinatorial structure.
• To develop a low-complexity, model-free algorithm for recovering discrete labels from noisy modulo differences.
• To prove that the proposed method converges to the maximum likelihood estimate under broad statistical models.
• To establish theoretical guarantees for exact recovery in a high-probability regime with minimal sample complexity.

Proposed method

• Lifts discrete label variables into a continuous space by representing distinct values in orthogonal directions.
• Represents the joint alignment problem as a quadratic optimization over hypercubes in the lifted space.
• Uses a spectral method for initialization to obtain a first guess within a basin of attraction of the global optimum.
• Applies projected power iterations to successively refine the estimate, ensuring iterates remain within the feasible hypercube.
• Employs a two-stage paradigm: spectral initialization followed by iterative refinement via projected power steps.
• Proves convergence to the maximum likelihood estimate by leveraging benign geometric structure and moderate deviation principles.

Experimental results

Research questions

• RQ1 Can an efficient algorithm be designed for joint alignment from noisy pairwise modulo differences when the label space is discrete and nonconvex?
• RQ2 Does a projected power method with spectral initialization achieve exact recovery in high probability under minimal sample complexity?
• RQ3 What is the minimal sample complexity required for the algorithm to succeed in recovering the true discrete labels?
• RQ4 How does the algorithm’s performance depend on the statistical model of the noise in pairwise differences?
• RQ5 Can the algorithm be proven to converge to the maximum likelihood estimate despite the nonconvexity of the discrete assignment problem?

Key findings

• The Projected Power Method achieves exact recovery (i.e., converges to the maximum likelihood estimate) with high probability under broad statistical models.
• Exact recovery is guaranteed when the number of observed pairwise differences satisfies $ p_{\text{obs}} \gtrsim \frac{\log^5(mn)}{n} $, where $ m $ is the number of label values and $ n $ the number of variables.
• The algorithm succeeds even when only 50% of the pairwise measurements are consistent with the true differences.
• Theoretical analysis shows that the method benefits from a large basin of attraction around the global optimum, enabling fast geometric convergence after spectral initialization.
• The method is model-free and does not require strong parametric assumptions on the noise distribution beyond moment conditions.
• Theoretical guarantees are established via moderate deviation principles and bounds on KL divergence and variance, showing the algorithm's robustness to noise.