[Paper Review] Taylor approximation for chance constrained optimization problems governed by partial differential equations with high-dimensional random parameters

Peng Chen, Omar Ghattas|arXiv (Cornell University)|Nov 19, 2020

Probabilistic and Robust Engineering Design79 references4 citations

TL;DR

This paper proposes a scalable Taylor approximation-based method for solving chance-constrained PDE-constrained optimization problems with high-dimensional random parameters. By leveraging high-order Taylor expansions, randomized Hessian approximation, and a continuation optimization scheme with smooth penalty methods, the approach reduces PDE solves by up to 37× while maintaining high accuracy, demonstrating scalability up to 100,000-dimensional parameter spaces in groundwater management tests.

ABSTRACT

We propose a fast and scalable optimization method to solve chance or probabilistic constrained optimization problems governed by partial differential equations (PDEs) with high-dimensional random parameters. To address the critical computational challenges of expensive PDE solution and high-dimensional uncertainty, we construct surrogates of the constraint function by Taylor approximation, which relies on efficient computation of the derivatives, low rank approximation of the Hessian, and a randomized algorithm for eigenvalue decomposition. To tackle the difficulty of the non-differentiability of the inequality chance constraint, we use a smooth approximation of the discontinuous indicator function involved in the chance constraint, and apply a penalty method to transform the inequality constrained optimization problem to an unconstrained one. Moreover, we design a gradient-based optimization scheme that gradually increases smoothing and penalty parameters to achieve convergence, for which we present an efficient computation of the gradient of the approximate cost functional by the Taylor approximation. Based on numerical experiments for a problem in optimal groundwater management, we demonstrate the accuracy of the Taylor approximation, its ability to greatly accelerate constraint evaluations, the convergence of the continuation optimization scheme, and the scalability of the proposed method in terms of the number of PDE solves with increasing random parameter dimension from one thousand to hundreds of thousands.

Motivation & Objective

• To address the computational intractability of chance-constrained PDE optimization with high-dimensional random parameters.
• To reduce the number of expensive PDE solves required for statistical estimation of chance constraints.
• To enable scalable optimization in high-dimensional parameter spaces where traditional sampling-based methods become infeasible.
• To develop a gradient-based optimization scheme that converges reliably despite the non-differentiability of chance constraints.
• To achieve high accuracy and efficiency through high-order Taylor approximations and randomized low-rank Hessian computation.

Proposed method

• Constructs surrogate models of the constraint function using Taylor approximations up to quadratic order, leveraging efficient derivative computation and low-rank Hessian approximation.
• Employs a smooth approximation of the discontinuous indicator function in the chance constraint to enable gradient-based optimization.
• Applies a continuation method with gradually increasing smoothing and penalty parameters to ensure convergence of the BFGS optimization scheme.
• Uses randomized algorithms for efficient eigenvalue decomposition of the Hessian, enabling scalable computation in high dimensions.
• Derives the gradient of the approximate cost functional via Taylor expansion to enable efficient optimization.
• Combines the Taylor approximation with a penalty method to transform the inequality-constrained problem into an unconstrained one for gradient-based solvers.

Figure 1: Left: Smooth approximation of the discontinuous indicator function ${\mathbb{I}}_{[0,\infty)}(x)$ by a logistic function $\ell_{\beta}(x)=\frac{1}{1+e^{-2\beta x}}$ with $\beta>0$ . Right: Penalty function $\mathcal{S}_{\gamma}(x)=\frac{\gamma}{2}(\max\{0,x\})^{2}$ with $\gamma>0$ .

Experimental results

Research questions

• RQ1 Can high-order Taylor approximations effectively and efficiently surrogate expensive chance constraints in PDE-constrained optimization with high-dimensional uncertainty?
• RQ2 How does the proposed method scale with increasing dimension of the random parameter space in terms of PDE solves and optimization iterations?
• RQ3 Can a continuation optimization scheme with smooth penalty and adaptive parameters achieve convergence while maintaining accuracy?
• RQ4 What is the trade-off between approximation accuracy and computational cost when using constant, linear, or quadratic Taylor approximations?
• RQ5 To what extent can randomized Hessian approximation and low-rank decomposition maintain accuracy while reducing computational cost?

Key findings

• The quadratic Taylor approximation achieves two orders of magnitude higher accuracy than sample average approximation (SAA) with 1024 samples, while requiring only 1/37th the number of PDE solves.
• The method reduces the number of PDE solves by a speedup factor of approximately 37× compared to SAA, with higher speedup for more expensive PDEs or larger sample counts.
• The number of PDE solves remains essentially constant as the random parameter dimension increases from 1,089 to 100,000, demonstrating strong scalability.
• The optimization convergence is robust and stable across increasing smoothing and penalty parameters, with consistent iteration counts.
• The eigenvalue decay of the Hessian approximation remains similar across different dimensions, confirming the scalability of the low-rank approximation.
• The optimal design obtained via quadratic Taylor approximation closely matches that from SAA, both in variable values and resulting pressure field reduction in the region of interest.

Figure 2: Groundwater flow in physical domain $(0,1)^{2}$ . Blue dots stand for the location of the extraction wells. A velocity field $\mathbf{v}$ is shown at the mean of the log-permeability $m=\bar{m}$ and optimal variable $z^{*}$ , which is obtained with quadratic approximation of the constraint