[Paper Review] Convergence Rate Analysis of a Stochastic Trust Region Method via Submartingales

José Blanchet, Coralia Cartis+2 more|arXiv (Cornell University)|Sep 23, 2016

Stochastic Gradient Optimization Techniques22 references15 citations

TL;DR

This paper introduces a novel stochastic process framework using supermartingales to analyze convergence rates of trust-region methods under noisy, adaptive estimates. It establishes the first global complexity bound of $O(\epsilon^{-2})$ for a stochastic trust-region method achieving $\|\nabla f(x)\| \leq \epsilon$, and extends this to $O(\epsilon^{-3})$ for second-order convergence under additional assumptions.

ABSTRACT

We propose a novel framework for analyzing convergence rates of stochastic optimization algorithms with adaptive step sizes. This framework is based on analyzing properties of an underlying generic stochastic process, in particular by deriving a bound on the expected stopping time of this process. We utilize this framework to analyze the bounds on expected global convergence rates of a stochastic variant of a traditional trust region method, introduced in \cite{ChenMenickellyScheinberg2014}. While traditional trust region methods rely on exact computations of the gradient, Hessian and values of the objective function, this method assumes that these values are available up to some dynamically adjusted accuracy. Moreover, this accuracy is assumed to hold only with some sufficiently large, but fixed, probability, without any additional restrictions on the variance of the errors. This setting applies, for example, to standard stochastic optimization and machine learning formulations. Improving upon the analysis in \cite{ChenMenickellyScheinberg2014}, we show that the stochastic process defined by the algorithm satisfies the assumptions of our proposed general framework, with the stopping time defined as reaching accuracy $\| abla f(x)\|\leq ε$. The resulting bound for this stopping time is $O(ε^{-2})$, under the assumption of sufficiently accurate stochastic gradient, and is the first global complexity bound for a stochastic trust-region method. Finally, we apply the same framework to derive second order complexity bound under some additional assumptions.

Motivation & Objective

• To develop a general framework for analyzing convergence rates of stochastic optimization algorithms with adaptive step sizes using stochastic processes.
• To analyze the expected global convergence rate of a stochastic trust-region method that uses dynamically accurate gradient and Hessian estimates.
• To establish the first global complexity bound for a stochastic trust-region method under general noise assumptions without variance restrictions.
• To extend the framework to derive second-order complexity bounds under additional regularity assumptions.
• To demonstrate that the method achieves convergence to second-order stationary points with complexity matching the deterministic case.

Proposed method

• Formulates a stochastic trust-region method where gradient, Hessian, and objective values are estimated with accuracy that improves adaptively over time.
• Applies a supermartingale-based framework to bound the expected stopping time of the stochastic process defined by the algorithm.
• Defines the stopping time as reaching $\|\nabla f(x)\| \leq \epsilon$, corresponding to first-order stationarity.
• Derives a bound on the expected decrease in a Lyapunov-type function $\Phi_k$ using constants derived from Lipschitz constants and model accuracy parameters.
• Uses assumptions on the accuracy of stochastic estimates (e.g., $\epsilon_F$, $\eta_2$) to control model quality and ensure convergence.
• Applies the framework to both first- and second-order cases, deriving complexity bounds via the expected stopping time of the process.

Experimental results

Research questions

• RQ1 Can a general stochastic process framework be developed to analyze convergence rates of adaptive stochastic optimization algorithms?
• RQ2 What is the global convergence complexity of a stochastic trust-region method when gradient and Hessian estimates are noisy but progressively accurate?
• RQ3 Does the stochastic trust-region method achieve the same convergence rate as deterministic trust-region methods under general noise conditions?
• RQ4 Can the framework be extended to establish second-order complexity bounds for convergence to second-order stationary points?
• RQ5 How does the adaptive accuracy of stochastic estimates affect the convergence rate and overall complexity of the algorithm?

Key findings

• The proposed framework establishes the first global complexity bound of $O(\epsilon^{-2})$ for a stochastic trust-region method under general noise assumptions.
• The expected stopping time $\mathbb{E}[T_\epsilon]$ for achieving $\|\nabla f(x)\| \leq \epsilon$ is bounded by $O(\epsilon^{-2})$ when gradient estimates are sufficiently accurate.
• For second-order convergence, the expected stopping time is bounded by $O(\epsilon^{-3})$, matching the deterministic case under additional assumptions.
• The complexity bound is derived via supermartingale analysis of a Lyapunov function $\Phi_k$, with explicit constants depending on Lipschitz constants and model accuracy parameters.
• The method does not require full gradient computation, making it suitable for purely stochastic and large-scale machine learning settings.
• The framework is general and has already been applied to other algorithms, such as stochastic line search, demonstrating broad applicability.