[Paper Review] A Local Analysis of Block Coordinate Descent for Gaussian Phase Retrieval

David A. Barmherzig, Ju Sun|arXiv (Cornell University)|Dec 6, 2017

Optical measurement and interference techniques11 references3 citations

TL;DR

This paper analyzes a block coordinate descent (BCD) algorithm for Gaussian phase retrieval, showing it converges linearly to the global minimizer when initialized near the solution. By fixing the dual variable to zero in ADMM—justified since the dual is zero at optimality—the BCD method becomes equivalent to solving a biconvex least-squares problem, and convergence is proven under deterministic initialization with a linear rate.

ABSTRACT

While convergence of the Alternating Direction Method of Multipliers (ADMM) on convex problems is well studied, convergence on nonconvex problems is only partially understood. In this paper, we consider the Gaussian phase retrieval problem, formulated as a linear constrained optimization problem with a biconvex objective. The particular structure allows for a novel application of the ADMM. It can be shown that the dual variable is zero at the global minimizer. This motivates the analysis of a block coordinate descent algorithm, which is equivalent to the ADMM with the dual variable fixed to be zero. We show that the block coordinate descent algorithm converges to the global minimizer at a linear rate, when starting from a deterministically achievable initialization point.

Motivation & Objective

• To understand the local convergence behavior of ADMM on nonconvex problems, using Gaussian phase retrieval as a model.
• To justify fixing the dual variable to zero in ADMM by showing it is zero at the global minimizer in the Gaussian phase retrieval problem.
• To analyze the convergence of a block coordinate descent (BCD) algorithm equivalent to ADMM with zero dual variable.
• To establish linear convergence of BCD to the global minimizer when initialized within a neighborhood of the solution.
• To show that a spectral initialization achieves the required initialization point deterministically.

Proposed method

• Formulates the Gaussian phase retrieval problem as a biconvex optimization problem with variable splitting: minimize $ f(\mathbf{z}, \mathbf{w}) = \frac{1}{4m}\sum_{k=1}^m (y_k^2 - \mathbf{a}_k^T \mathbf{z} \mathbf{a}_k^T \mathbf{w})^2 $ subject to $ \mathbf{z} = \mathbf{w} $.
• Derives the augmented Lagrangian for the ADMM algorithm, showing that at critical points, the dual variable $ \mathbf{\lambda} = \mathbf{0} $ at the global minimizer.
• Proposes a block coordinate descent (BCD) algorithm equivalent to ADMM with $ \mathbf{\lambda} $ fixed at zero, updating $ \mathbf{z} $ and $ \mathbf{w} $ sequentially.
• Analyzes the BCD update in expectation over the random Gaussian measurement vectors, focusing on local convergence near the true signal.
• Uses a spectral initialization to achieve a deterministic initialization point within the required neighborhood of the global minimizer.
• Employs a contraction argument based on inner products and norms to prove that each BCD step reduces the distance to the true signal by a constant factor, establishing linear convergence.

Experimental results

Research questions

• RQ1 Can the ADMM algorithm be rigorously analyzed for local convergence on nonconvex problems like Gaussian phase retrieval?
• RQ2 Is the dual variable zero at the global minimizer in the Gaussian phase retrieval problem, enabling a simplified BCD approach?
• RQ3 Does block coordinate descent with fixed dual variable converge linearly to the global minimizer when initialized near the solution?
• RQ4 Can a deterministic spectral initialization achieve the required initialization neighborhood for linear convergence?
• RQ5 What is the rate of convergence of the BCD algorithm in expectation over the random measurement vectors?

Key findings

• The dual variable $ \mathbf{\lambda} $ is zero at the global minimizer of the Gaussian phase retrieval problem, justifying its fixed-zero setting.
• The block coordinate descent algorithm, equivalent to ADMM with zero dual variable, converges to the global minimizer at a linear rate.
• Linear convergence is established when the algorithm starts within a neighborhood of the true signal, specifically when $ \|\mathbf{w} - \mathbf{x}\| \leq \frac{1}{8}\|\mathbf{x}\| $.
• The required initialization can be achieved deterministically via spectral decomposition, as shown in prior work.
• The convergence rate is quantified by showing that each BCD step reduces the distance to the solution by a factor bounded away from one, with the contraction factor depending on the initialization quality.
• The analysis shows that for sufficiently large penalty parameter $ \rho = C\|\mathbf{x}\|^2 $ with $ C \leq 27/8 $, the BCD iterates remain in the neighborhood of the global minimizer and converge linearly.