[Paper Review] Inexact Sequential Quadratic Optimization for Minimizing a Stochastic Objective Function Subject to Deterministic Nonlinear Equality Constraints

Frank E. Curtis, Daniel P. Robinson+1 more|arXiv (Cornell University)|Jul 7, 2021

Stochastic Gradient Optimization Techniques39 references4 citations

TL;DR

This paper proposes an inexact stochastic sequential quadratic optimization (SISQO) algorithm for minimizing a stochastic objective subject to deterministic nonlinear equality constraints, where only noisy gradient estimates are available. By allowing inexact solutions to subproblems using iterative linear solvers like MINRES, the method reduces computational cost while maintaining convergence in expectation, outperforming exact subproblem methods in numerical tests.

ABSTRACT

An algorithm is proposed, analyzed, and tested experimentally for solving stochastic optimization problems in which the decision variables are constrained to satisfy equations defined by deterministic, smooth, and nonlinear functions. It is assumed that constraint function and derivative values can be computed, but that only stochastic approximations are available for the objective function and its derivatives. The algorithm is of the sequential quadratic optimization variety. A distinguishing feature of the algorithm is that it allows inexact subproblem solutions to be employed, which is particularly useful in large-scale settings when the matrices defining the subproblems are too large to form and/or factorize. Conditions are imposed on the inexact subproblem solutions that account for the fact that only stochastic objective gradient estimates are available. Convergence results in expectation are established for the method. Numerical experiments show that it outperforms an alternative algorithm that employs highly accurate subproblem solutions in every iteration.

Motivation & Objective

• To develop an efficient algorithm for solving stochastic optimization problems with deterministic nonlinear equality constraints when only noisy gradient estimates are available.
• To extend sequential quadratic programming (SQP) methods to the stochastic setting with inexact subproblem solutions, improving scalability for large-scale problems.
• To establish convergence in expectation under inexactness conditions tailored to stochastic gradient approximations.
• To demonstrate empirically that inexact subproblem solutions yield better overall performance than exact solutions in terms of iterations and computational cost.
• To provide a practical algorithm that balances per-iteration cost and global convergence for real-world applications involving uncertainty and constraints.

Proposed method

• The algorithm uses sequential quadratic programming (SQP) to solve stochastic optimization problems with deterministic nonlinear equality constraints.
• It employs stochastic gradient estimates for the objective function and its derivatives, while constraint functions and their derivatives are computed exactly.
• Inexact solutions to the quadratic subproblems are allowed, with convergence conditions adapted to account for stochastic gradient noise.
• An iterative linear solver (e.g., MINRES) is used to solve the indefinite linear systems arising in each SQP subproblem, avoiding full matrix factorization.
• The method enforces inexactness tolerances that depend on the noise level in the stochastic gradients, ensuring sufficient progress toward optimality.
• A line search on a merit function is used to determine step sizes, maintaining global convergence properties under appropriate conditions.

Experimental results

Research questions

• RQ1 Can an inexact SQP method be designed and analyzed for stochastic optimization problems with deterministic nonlinear equality constraints?
• RQ2 What conditions on inexact subproblem solutions ensure convergence in expectation when only stochastic gradients are available?
• RQ3 Does allowing inexact subproblem solutions lead to better overall performance compared to exact subproblem solutions in large-scale settings?
• RQ4 How do the inexactness tolerances relate to the noise level in stochastic gradient estimates?
• RQ5 Can the proposed method outperform exact subproblem solvers in terms of total computational cost and convergence speed?

Key findings

• SISQO achieved better average feasibility and stationarity errors than the exact subproblem variant (SISQO_exact), despite performing more outer iterations.
• For problem (47), SISQO reduced feasibility error by over 90% compared to SISQO_exact when εN = 10⁻⁴, with 8.9 MINRES iterations per outer iteration.
• For problem (48), SISQO achieved a feasibility error of 1.68×10⁻⁷ with 14.5 MINRES iterations per outer iteration, while SISQO_exact had a 1.31×10⁻⁵ feasibility error.
• SISQO required fewer total MINRES iterations than SISQO_exact across all test cases, indicating better efficiency in solving the linear systems.
• The method maintained convergence in expectation under inexactness conditions tailored to stochastic gradient noise, matching theoretical guarantees of exact methods.
• Numerical results show that inexactness leads to better trade-offs between per-iteration cost and global convergence, especially in large-scale problems.