[Paper Review] Convergence Certificate for Stochastic Derivative-Free Trust-Region Methods based on Gaussian Processes

Harsh Shukla, Tafarel de Avila Ferreira+3 more|arXiv (Cornell University)|Oct 2, 2020

Gaussian Processes and Bayesian Inference19 references4 citations

TL;DR

This paper proves that Gaussian Processes (GPs) are probabilistically fully linear, enabling global convergence of stochastic derivative-free trust-region methods. By certifying GP surrogates as fully linear, the method ensures fast, globally convergent optimization with fewer plant evaluations, demonstrated on a chemical reactor batch process with noisy, uncertain dynamics.

ABSTRACT

In many machine learning applications, one wants to learn the unknown objective and constraint functions of an optimization problem from available data and then apply some technique to attain a local optimizer of the learned model. This work considers Gaussian processes as global surrogate models and utilizes them in conjunction with derivative-free trust-region methods. It is well known that derivative-free trust-region methods converge globally---provided the surrogate model is probabilistically fully linear. We prove that \glspl{gp} are indeed probabilistically fully linear, thus resulting in fast (compared to linear or quadratic local surrogate models) and global convergence. We draw upon the optimization of a chemical reactor to demonstrate the efficiency of \gls{gp}-based trust-region methods.

Motivation & Objective

• Address the lack of theoretical convergence guarantees for GP-based derivative-free optimization by certifying their fully-linear properties.
• Enable global convergence of trust-region methods when using GPs as surrogate models for unknown or noisy objectives.
• Reduce the number of expensive plant evaluations in real-time optimization by leveraging GP surrogates with certified accuracy.
• Demonstrate the method's effectiveness in a noisy, uncertain chemical batch reactor process with structural model mismatch.
• Provide a theoretical foundation for using GPs in stochastic, derivative-free trust-region optimization with almost-sure convergence guarantees.

Proposed method

• Use Gaussian Processes (GPs) as global surrogate models to approximate unknown, noisy objective and constraint functions.
• Apply a probabilistic derivative-free trust-region method that iteratively refines the trust region based on GP predictions and uncertainty.
• Certify that GPs are probabilistically fully linear by proving bounds on the approximation error in the trust region.
• Incorporate new data points into the GP model only if they are sufficiently far from existing points to prevent overfitting.
• Use a trust-region algorithm with adaptive radius adjustment (based on η, γ_dec, γ_inc, Δ₀) to ensure convergence.
• Reformulate constrained optimization problems by penalizing constraint violations in the cost function, enabling unconstrained optimization.

Experimental results

Research questions

• RQ1 Can Gaussian Processes be formally certified as probabilistically fully linear, a key requirement for global convergence of trust-region methods?
• RQ2 Does using GPs as global surrogate models in derivative-free trust-region optimization lead to faster convergence and fewer function evaluations compared to local models?
• RQ3 How well do GP-based trust-region methods perform in the presence of model mismatch and noisy measurements in real-world optimization problems?
• RQ4 Can the method adapt to sudden changes in system dynamics without retraining the GP from scratch?
• RQ5 What is the theoretical convergence guarantee for stochastic, derivative-free trust-region methods when using GP surrogates?

Key findings

• The paper proves that Gaussian Processes are probabilistically fully linear, which establishes the theoretical foundation for global convergence of trust-region methods using GPs.
• The GP-based trust-region method achieves near-plant optimality within just three iterations in the chemical reactor case study, despite model mismatch and noise.
• The method significantly outperforms model-based optimization in Scenario II, where the true reaction kinetics differ from the assumed model, due to the GP's ability to learn new dynamics.
• The GP model adapts quickly to changes in system behavior (e.g., from Scenario I to II) without reinitialization, maintaining high performance.
• Fewer trust-region iterations are required when using GPs compared to local surrogate models, reducing the number of expensive plant evaluations.
• The algorithm maintains feasibility of constraints and converges to a solution close to the true plant optimum, even with 5% additive Gaussian noise.