[Paper Review] Convergence properties of gradient methods for blind ptychography

Oleh Melnyk|arXiv (Cornell University)|Jun 14, 2023

Advanced X-ray Imaging Techniques4 citations

TL;DR

This paper establishes the first convergence guarantees for gradient and stochastic gradient descent methods in blind ptychography, proving sublinear convergence to a critical point under proper step size selection. It formally links the extended Ptychographic Iterative Engine (ePIE) to stochastic gradient descent and provides theoretical justification for its empirical success.

ABSTRACT

We consider blind ptychography, an imaging technique which aims to reconstruct an object of interest from a set of its diffraction patterns, each obtained by a local illumination. As the distribution of the light within the illuminated region, called the window, is unknown, it also has to be estimated as well. For the recovery, we consider gradient and stochastic gradient descent methods for the minimization of amplitude-base squared loss. In particular, this includes extended Ptychographic Iterative Engine as a special case of stochastic gradient descent. We show that all methods converge to a critical point at a sublinear rate with a proper choice of step sizes. We also discuss possibilities for larger step sizes.

Motivation & Objective

• To provide a rigorous mathematical convergence analysis for gradient-based methods in blind ptychography, a technique that jointly reconstructs an object and its illumination function from diffraction patterns.
• To establish theoretical justification for the empirical success of the extended Ptychographic Iterative Engine (ePIE), which lacks prior convergence guarantees.
• To analyze the convergence behavior of both standard and stochastic gradient descent under amplitude-based squared loss for blind ptychographic reconstruction.
• To explore the possibility of larger step sizes in gradient methods while maintaining convergence.

Proposed method

• Formulates blind ptychography as a joint optimization problem minimizing an amplitude-based squared loss function over both the object and the probe (window).
• Applies standard gradient descent and stochastic gradient descent (SGD) to minimize the loss, with updates based on individual diffraction patterns in the stochastic case.
• Derives convergence rates by analyzing the expected decrease in the loss function and bounding martingale terms arising from stochastic sampling.
• Uses tools from stochastic optimization, including Lyapunov functions and martingale convergence theorems, to prove convergence to a critical point.
• Establishes that the gradient norms of both object and probe parameters converge to zero almost surely under appropriate step size sequences.
• Connects the ePIE algorithm directly to SGD by showing its update rule matches that of stochastic gradient descent with specific sampling and step size choices.

Experimental results

Research questions

• RQ1 Does gradient descent converge for blind ptychography under amplitude-based squared loss?
• RQ2 Can the extended Ptychographic Iterative Engine (ePIE) be formally justified as a stochastic gradient descent method?
• RQ3 What step size conditions ensure sublinear convergence to a critical point in blind ptychography?
• RQ4 Are there theoretical bounds on larger step sizes that still preserve convergence?
• RQ5 Can the convergence of joint object-probe optimization be rigorously established without additional assumptions?

Key findings

• Gradient descent and stochastic gradient descent converge to a critical point of the loss function at a sublinear rate when step sizes are chosen appropriately, specifically with summable and square-summable sequences.
• The extended Ptychographic Iterative Engine (ePIE) is formally identified as a special case of stochastic gradient descent, providing the first theoretical convergence guarantee for this widely used algorithm.
• The gradient norms of both the object and probe parameters converge to zero almost surely, indicating convergence to a critical point.
• The analysis holds even when sampling indices are not i.i.d., provided the sampling scheme satisfies certain boundedness and mixing conditions.
• Larger step sizes are possible under additional assumptions, such as boundedness of the gradient norms and convergence of the sum of squared step sizes.
• The proof relies on martingale convergence and Lyapunov function techniques to control the expected decrease in the loss and the variance of the stochastic updates.