[Paper Review] On the convergence of mirror descent beyond stochastic convex programming

Zhengyuan Zhou, Panayotis Mertikopoulos+3 more|arXiv (Cornell University)|Jun 18, 2017

Sparse and Compressive Sensing Techniques37 references3 citations

TL;DR

This paper establishes almost sure convergence of stochastic mirror descent (SMD) to a global minimum under a weaker condition than convexity—variational coherence—demonstrating that SMD's last iterate converges with probability 1 even in non-convex problems. It further shows that in problems with sharp minima, SMD reaches a minimum in finite steps almost surely, despite persistent gradient noise.

ABSTRACT

In this paper, we examine the convergence of mirror descent in a class of stochastic optimization problems that are not necessarily convex (or even quasi-convex), and which we call variationally coherent. Since the standard technique of "ergodic averaging" offers no tangible benefits beyond convex programming, we focus directly on the algorithm's last generated sample (its "last iterate"), and we show that it converges with probabiility $1$ if the underlying problem is coherent. We further consider a localized version of variational coherence which ensures local convergence of stochastic mirror descent (SMD) with high probability. These results contribute to the landscape of non-convex stochastic optimization by showing that (quasi-)convexity is not essential for convergence to a global minimum: rather, variational coherence, a much weaker requirement, suffices. Finally, building on the above, we reveal an interesting insight regarding the convergence speed of SMD: in problems with sharp minima (such as generic linear programs or concave minimization problems), SMD reaches a minimum point in a finite number of steps (a.s.), even in the presence of persistent gradient noise. This result is to be contrasted with existing black-box convergence rate estimates that are only asymptotic.

Motivation & Objective

• To analyze the convergence of stochastic mirror descent (SMD) in non-convex optimization problems where standard convexity assumptions do not hold.
• To identify a weaker condition than convexity or quasi-convexity that still guarantees convergence of SMD’s last iterate to a global minimum.
• To extend convergence analysis to locally coherent problems where global convexity fails, particularly in multi-modal settings.
• To investigate the finite-time convergence behavior of SMD in problems with sharp minima under persistent gradient noise.
• To demonstrate that ergodic averaging is not necessary for convergence in non-convex settings, and that the last iterate suffices under variational coherence.

Proposed method

• Introduces the concept of 'variational coherence' as a generalization of monotonicity and convexity, enabling convergence analysis beyond convex programs.
• Analyzes the dual averaging (or 'lazy') variant of SMD, using a mirror map to project gradient updates into the feasible set.
• Employs a Fenchel coupling-based Lyapunov function to track progress toward the optimal set, leveraging its positive-definiteness and reciprocity properties.
• Applies martingale convergence theorems and the law of large numbers for martingale differences to show almost sure convergence of the last iterate.
• Introduces a localized version of variational coherence to handle non-convex, multi-modal problems where local minima are not locally convex.
• Uses bounded $L^2$-norm gradient samples and square-summable step-sizes to ensure stability and convergence in the presence of noise.

Experimental results

Research questions

• RQ1 Can stochastic mirror descent converge to a global minimum in non-convex problems without requiring convexity or quasi-convexity?
• RQ2 Does the last iterate of SMD converge almost surely under a weaker condition than convexity?
• RQ3 Can variational coherence, a weaker structural condition, ensure almost sure convergence of SMD’s last iterate?
• RQ4 What is the convergence behavior of SMD in problems with sharp minima under persistent gradient noise?
• RQ5 Can local convergence be guaranteed in non-convex problems where the minimum is not locally convex?

Key findings

• Under variational coherence, the last iterate of stochastic mirror descent converges to a global minimum with probability 1, even in non-convex problems.
• The convergence is established without ergodic averaging, directly analyzing the last generated sample, which is more natural in non-convex settings.
• In problems with sharp minima—such as generic linear programs or concave minimization—SMD reaches a minimum point in a finite number of steps almost surely, despite persistent gradient noise.
• The localized version of variational coherence ensures local convergence of SMD with high probability, even when the minimum is not locally convex.
• The analysis reveals that (quasi-)convexity is not essential for convergence; variational coherence is a sufficient and strictly weaker condition.
• The Fenchel coupling is used as a Lyapunov function, and its recurrence properties are leveraged to show that the iterates enter and remain near the optimal set infinitely often almost surely.