[Paper Review] Reproducibility in Optimization: Theoretical Framework and Limits

Kwangjun Ahn, Prateek Jain+4 more|arXiv (Cornell University)|Feb 9, 2022

Stochastic Gradient Optimization Techniques5 citations

TL;DR

This paper establishes a theoretical framework for quantifying reproducibility limits in convex optimization under noisy or inexact operations such as stochastic gradients and numerical errors. It proves a fundamental trade-off between computational cost and reproducibility, showing that lower step-sizes and more iterations are necessary and sufficient to achieve optimal reproducibility, with tight bounds derived for smooth, non-smooth, and strongly-convex settings.

ABSTRACT

We initiate a formal study of reproducibility in optimization. We define a quantitative measure of reproducibility of optimization procedures in the face of noisy or error-prone operations such as inexact or stochastic gradient computations or inexact initialization. We then analyze several convex optimization settings of interest such as smooth, non-smooth, and strongly-convex objective functions and establish tight bounds on the limits of reproducibility in each setting. Our analysis reveals a fundamental trade-off between computation and reproducibility: more computation is necessary (and sufficient) for better reproducibility.

Motivation & Objective

• To formalize the fundamental limits of reproducibility in convex optimization under noisy or inexact operations such as stochastic gradients and numerical errors.
• To analyze reproducibility in key convex optimization settings: smooth, non-smooth, strongly-convex, finite-sum, and stochastic optimization.
• To establish a theoretical trade-off between computational complexity (number of iterations) and reproducibility.
• To determine whether practical first-order methods can achieve these theoretical limits.

Proposed method

• Introduces a quantitative measure of reproducibility based on the expected squared distance between two optimization trajectories under identical hyperparameters but different noise realizations.
• Analyzes projected gradient descent with constant step-size in a bounded domain to derive convergence and deviation bounds under gradient noise.
• Uses recursive inequalities and telescoping sums to bound the expected squared deviation between two trajectories, incorporating gradient noise and initialization errors.
• Derives upper bounds on deviation by choosing small step-sizes and large iteration counts, showing that such settings achieve the theoretical reproducibility limits.
• Applies the framework to multiple convex settings, including smooth, non-smooth, and strongly-convex functions, deriving matching lower and upper bounds.
• Employs concentration and martingale-style arguments to control the propagation of errors across iterations, leading to tight deviation bounds.

Experimental results

Research questions

• RQ1 What are the fundamental limits of reproducibility in convex optimization under inexact gradient or initialization noise?
• RQ2 How does the trade-off between computational cost and reproducibility manifest across different convex optimization settings?
• RQ3 Can standard first-order methods achieve the theoretical limits of reproducibility?
• RQ4 What role do step-size and iteration count play in achieving optimal reproducibility?
• RQ5 How do gradient noise and numerical errors propagate through iterative optimization, and what bounds can be placed on their impact?

Key findings

• A fundamental trade-off exists between computation and reproducibility: more iterations (higher computation) are necessary for better reproducibility, even in convex settings.
• For ε-accuracy, the expected squared deviation between two trajectories scales as O(1/(ε²T) + δ²/ε²), where T is the number of iterations and δ is the gradient noise level.
• Projected gradient descent with small step-sizes and large iteration counts achieves the theoretical reproducibility limits in smooth, non-smooth, and strongly-convex settings.
• The lower bound on reproducibility matches the upper bound achieved by standard gradient descent, proving tightness of the limits.
• Even with fixed random seeds, irreproducibility persists due to hardware-level nondeterminism and numerical errors, which cannot be fully eliminated.
• The results suggest that advanced techniques like regularization do not improve reproducibility beyond what is achievable with simple gradient descent under optimal step-size tuning.