[Paper Review] Deep S$^3$PR: Simultaneous Source Separation and Phase Retrieval Using Deep Generative Models

Christopher A. Metzler, Gordon Wetzstein|arXiv (Cornell University)|Feb 13, 2020

Underwater Acoustics Research4 citations

TL;DR

This paper proposes Deep S³PR, a novel method for solving the simultaneous source separation and phase retrieval (S³PR) problem by leveraging deep generative models as priors. By constraining source signals to lie within the latent space of a pretrained generator, the method efficiently solves the highly underdetermined and non-convex S³PR problem through latent variable optimization, achieving robust reconstruction of multiple images from intensity-only measurements across Gaussian, CDP, and Fourier forward models.

ABSTRACT

This paper introduces and solves the simultaneous source separation and phase retrieval (S$^3$PR) problem. S$^3$PR is an important but largely unsolved problem in a number application domains, including microscopy, wireless communication, and imaging through scattering media, where one has multiple independent coherent sources whose phase is difficult to measure. In general, S$^3$PR is highly under-determined, non-convex, and difficult to solve. In this work, we demonstrate that by restricting the solutions to lie in the range of a deep generative model, we can constrain the search space sufficiently to solve S$^3$PR.

Motivation & Objective

• To formalize and solve the simultaneous source separation and phase retrieval (S³PR) problem, which is highly underdetermined and non-convex in applications like scattering media imaging and coherent microscopy.
• To address the failure of conventional cascaded approaches (source separation followed by phase retrieval) due to high similarity between intensity components from different sources.
• To demonstrate that deep generative models provide strong, realistic priors that constrain the solution space sufficiently to enable stable and accurate S³PR recovery.
• To generalize the solution across diverse forward models (Gaussian, CDP, Fourier) and mixture sizes without retraining.
• To establish a foundation for future physical validation in applications such as imaging through scattering media and around corners.

Proposed method

• The method formulates S³PR as an optimization over the latent codes of a deep generative model G: R^k → R^n, where k ≪ n, to reconstruct signals x_l = G(z_l).
• It solves the problem by minimizing the data-fidelity loss: ||y - ∑_l |A G(z_l)|²||²₂ using alternating descent on the latent variables z_1, ..., z_L.
• For Fourier measurements, the optimization is reformulated in the Fourier domain using Parseval’s theorem to improve convergence: ||F⁻¹y - ∑_l G(z_l) ⋆ G(z_l)||²₂.
• The approach leverages the inductive bias of pretrained generative models (e.g., on MNIST and Fashion-MNIST) to regularize the solution space and avoid local minima.
• The method is agnostic to the forward model and generalizes across measurement types (Gaussian, CDP, Fourier) without architectural changes.
• The optimization is initialized with random latent codes and refined iteratively using gradient descent.

Experimental results

Research questions

• RQ1 Can deep generative models serve as effective priors to solve the highly underdetermined and non-convex S³PR problem?
• RQ2 Does constraining solutions to the range of a generative model enable reliable separation of multiple coherent sources from intensity-only measurements?
• RQ3 How does the performance of Deep S³PR compare to cascaded source separation and phase retrieval (USS + PR) across different measurement models and noise levels?
• RQ4 To what extent does the method generalize across forward models (Gaussian, CDP, Fourier) and dataset types (MNIST, Fashion-MNIST)?
• RQ5 What are the quantitative limits of reconstruction accuracy in terms of NMSE when recovering multiple images under varying mixture sizes and SNR?

Key findings

• Deep S³PR achieves significantly lower NMSE than the cascaded USS + PR baseline across all measurement models, with a 2.5× improvement on average for 2× MNIST under Gaussian measurements.
• For Fourier measurements, Deep S³PR achieves an average NMSE of 0.28 ± 0.06 for four MNIST images, compared to 0.87 ± 0.12 for USS + PR, demonstrating robustness in a challenging, real-world setting.
• The method successfully reconstructs the general shape of Fashion-MNIST images with two sources under Fourier measurements, though artifacts appear in higher mixtures.
• With CDP measurements, Deep S³PR maintains low error (NMSE ≈ 0.14–0.28) even for four-image mixtures, outperforming the sequential approach which fails at two images.
• The method shows improved convergence and stability when the loss is reformulated in the Fourier domain using autocorrelation, particularly for Fourier measurements.
• Empirical results confirm that the sequential USS + PR approach fails even with two images under Fourier measurements, while Deep S³PR remains effective up to four images.