[Paper Review] Phase Retrieval Under a Generative Prior

Paul Hand, Oscar Leong+1 more|arXiv (Cornell University)|Jul 11, 2018

Advanced X-ray Imaging Techniques2 references16 citations

TL;DR

This paper proposes a novel phase retrieval framework using deep generative models as priors, optimizing over the latent space to recover signals with information-theoretically optimal sample complexity. It proves that gradient methods converge globally under mild conditions when using $ m = O(kd^2\log n) $ measurements, outperforming traditional $ \ell_1 $-based sparse recovery methods.

ABSTRACT

The phase retrieval problem asks to recover a natural signal $y_0 \in \mathbb{R}^n$ from $m$ quadratic observations, where $m$ is to be minimized. As is common in many imaging problems, natural signals are considered sparse with respect to a known basis, and the generic sparsity prior is enforced via $\ell_1$ regularization. While successful in the realm of linear inverse problems, such $\ell_1$ methods have encountered possibly fundamental limitations, as no computationally efficient algorithm for phase retrieval of a $k$-sparse signal has been proven to succeed with fewer than $O(k^2\log n)$ generic measurements, exceeding the theoretical optimum of $O(k \log n)$. In this paper, we propose a novel framework for phase retrieval by 1) modeling natural signals as being in the range of a deep generative neural network $G : \mathbb{R}^k ightarrow \mathbb{R}^n$ and 2) enforcing this prior directly by optimizing an empirical risk objective over the domain of the generator. Our formulation has provably favorable global geometry for gradient methods, as soon as $m = O(kd^2\log n)$, where $d$ is the depth of the network. Specifically, when suitable deterministic conditions on the generator and measurement matrix are met, we construct a descent direction for any point outside of a small neighborhood around the unique global minimizer and its negative multiple, and show that such conditions hold with high probability under Gaussian ensembles of multilayer fully-connected generator networks and measurement matrices. This formulation for structured phase retrieval thus has two advantages over sparsity based methods: 1) deep generative priors can more tightly represent natural signals and 2) information theoretically optimal sample complexity. We corroborate these results with experiments showing that exploiting generative models in phase retrieval tasks outperforms sparse phase retrieval methods.

Motivation & Objective

• To overcome the fundamental sample complexity limitations of $ \ell_1 $-regularized phase retrieval, which requires $ O(k^2\log n) $ measurements for $ k $-sparse signals.
• To develop a new phase retrieval framework that models natural signals as outputs of deep generative networks, enabling tighter representation and lower measurement requirements.
• To establish provable global convergence guarantees for gradient-based optimization in the latent space of generative models under generic measurement ensembles.
• To demonstrate that the proposed method achieves information-theoretically optimal sample complexity $ O(kd^2\log n) $, where $ d $ is the network depth.

Proposed method

• Formulates phase retrieval as a latent space optimization problem: minimize $ \frac{1}{2}\| |AG(x)| - |AG(x_0)| \|^2 $ over $ x \in \mathbb{R}^k $, where $ G $ is a deep ReLU-based generative model.
• Imposes a generative prior by restricting recovery to the range of a deep neural network $ G: \mathbb{R}^k \to \mathbb{R}^n $, parameterized by $ d $-layer fully connected weights with ReLU activations.
• Establishes deterministic conditions on $ G $ and $ A $ under which the empirical risk objective has a descent direction from any point outside a small neighborhood of the global minimizer and its negative.
• Proves that these conditions hold with high probability when $ G $ has i.i.d. $ \mathcal{N}(0,1/n_i) $ weights and $ A $ has i.i.d. $ \mathcal{N}(0,1/m) $ entries.
• Uses a restricted range concentration property (RRCP) to control the deviation of $ A^\top A $ from its expectation on the image of the generator's Jacobian.
• Applies a uniform restricted concentration property (RCP) to bound the number of sign patterns of $ Av $, ensuring stability in the optimization landscape.

Experimental results

Research questions

• RQ1 Can deep generative priors enable phase retrieval with information-theoretically optimal sample complexity?
• RQ2 Does the empirical risk objective over the latent space of a generative model have favorable global geometry for gradient descent?
• RQ3 Under what conditions on the generator and measurement matrix does the objective function admit a descent direction from any non-solution point?
• RQ4 Can the proposed framework outperform traditional $ \ell_1 $-based sparse phase retrieval in terms of sample efficiency and recovery accuracy?

Key findings

• The proposed Deep Phase Retrieval (DPR) framework achieves information-theoretically optimal sample complexity of $ O(kd^2\log n) $, where $ d $ is the network depth and $ k $ is the latent dimension.
• With high probability, the empirical risk objective has a descent direction from any point outside a small neighborhood of the true latent code $ x_0 $ and its negative, under Gaussian generator and measurement matrices.
• The deterministic conditions required for favorable geometry hold with high probability when the generator weights and measurement matrix are i.i.d. Gaussian, ensuring global convergence of gradient methods.
• The method outperforms $ \ell_1 $-based sparse phase retrieval in experiments, demonstrating tighter signal representation and lower measurement requirements.
• The analysis establishes that the number of sign patterns of $ Av $ for $ v $ in a $ k $-dimensional subspace is bounded by $ O(m^{2k}) $, enabling concentration control in the optimization landscape.