[Paper Review] Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems

Yuxin Chen, Emmanuel J. Candès|arXiv (Cornell University)|May 19, 2015

Soil Geostatistics and Mapping50 references89 citations

TL;DR

This paper proposes a novel nonconvex optimization algorithm for solving random quadratic systems of equations, leveraging adaptive thresholding and a tailored objective function to achieve linear-time convergence. It proves that under mild conditions, the method recovers the true solution with high probability when the number of equations exceeds a constant multiple of the number of variables, matching the computational complexity of solving linear systems.

ABSTRACT

We consider the fundamental problem of solving quadratic systems of equations in $n$ variables, where $y_i = |\\langle \\boldsymbol{a}_i, \\boldsymbol{x} \ angle|^2$, $i = 1, \\ldots, m$ and $\\boldsymbol{x} \\in \\mathbb{R}^n$ is unknown. We propose a novel method, which starting with an initial guess computed by means of a spectral method, proceeds by minimizing a nonconvex functional as in the Wirtinger flow approach. There are several key distinguishing features, most notably, a distinct objective functional and novel update rules, which operate in an adaptive fashion and drop terms bearing too much influence on the search direction. These careful selection rules provide a tighter initial guess, better descent directions, and thus enhanced practical performance. On the theoretical side, we prove that for certain unstructured models of quadratic systems, our algorithms return the correct solution in linear time, i.e. in time proportional to reading the data $\\{\\boldsymbol{a}_i\\}$ and $\\{y_i\\}$ as soon as the ratio $m/n$ between the number of equations and unknowns exceeds a fixed numerical constant. We extend the theory to deal with noisy systems in which we only have $y_i \\approx |\\langle \\boldsymbol{a}_i, \\boldsymbol{x} \ angle|^2$ and prove that our algorithms achieve a statistical accuracy, which is nearly un-improvable. We complement our theoretical study with numerical examples showing that solving random quadratic systems is both computationally and statistically not much harder than solving linear systems of the same size---hence the title of this paper. For instance, we demonstrate empirically that the computational cost of our algorithm is about four times that of solving a least-squares problem of the same size.

Motivation & Objective

• To address the computational challenge of solving quadratic systems where only magnitude measurements are available, a common issue in phase retrieval and signal recovery.
• To develop an efficient, nonconvex optimization method that avoids the computational intractability of maximum likelihood estimation in such systems.
• To achieve linear-time convergence for random quadratic systems under minimal assumptions on the design vectors.
• To extend the method to noisy settings, ensuring statistical accuracy close to the theoretical limit.

Proposed method

• The algorithm begins with a spectral initialization to obtain a good initial guess for the solution.
• It employs a nonconvex objective function tailored to the quadratic system structure, differing from prior approaches like Wirtinger flow.
• Adaptive update rules dynamically drop terms that could unduly influence the search direction, improving convergence stability.
• A backtracking line search ensures sufficient decrease in the objective at each iteration, with theoretical guarantees on step size selection.
• The method uses truncated empirical risk minimization, focusing on terms that contribute most reliably to descent.
• Theoretical analysis relies on concentration inequalities and careful control of error terms in high-dimensional random matrix theory.

Experimental results

Research questions

• RQ1 Can random quadratic systems be solved efficiently in practice, approaching the computational cost of linear systems?
• RQ2 Does a nonconvex optimization approach with adaptive thresholding outperform standard methods in terms of convergence speed and accuracy?
• RQ3 What is the minimal number of equations required to recover the true solution with high probability in random quadratic systems?
• RQ4 How does the algorithm perform under noisy measurements, particularly in the Poisson model common in imaging applications?
• RQ5 Can the method achieve statistical accuracy that is nearly unimprovable under noisy conditions?

Key findings

• The algorithm recovers the true solution in linear time, i.e., in time proportional to reading the data, when the ratio of equations to variables exceeds a fixed numerical constant.
• For noisy systems, the method achieves statistical accuracy that is nearly unimprovable, matching the Cramér-Rao lower bound up to logarithmic factors.
• Empirically, the computational cost is about four times that of solving a least-squares problem of the same size.
• Theoretical analysis shows that the algorithm converges to the true solution with high probability under mild random design assumptions.
• Adaptive thresholding and objective function design lead to tighter initial guesses and better descent directions, enhancing practical performance.