[Paper Review] Turnpike in Lipschitz-nonlinear optimal control

Carlos Esteve, Borjan Geshkovski+2 more|arXiv (Cornell University)|Nov 22, 2020

Model Reduction and Neural Networks44 references4 citations

TL;DR

This paper presents a novel proof of the turnpike property for nonlinear optimal control problems with globally Lipschitz (possibly nonsmooth) nonlinearities, using a controllability-based construction of quasi-turnpike controls and a bootstrap argument. The method establishes exponential turnpike estimates without requiring linearization, smallness assumptions on initial data or targets, or smoothness of the nonlinearity, and applies to finite- and infinite-dimensional systems including ResNets and semilinear PDEs like wave and heat equations.

ABSTRACT

We present a new proof of the turnpike property for nonlinear optimal control problems, when the running target is a steady control-state pair of the underlying system. Our strategy combines the construction of quasi-turnpike controls via controllability, and a bootstrap argument, and does not rely on analyzing the optimality system or linearization techniques. This in turn allows us to address several optimal control problems for finite-dimensional, control-affine systems with globally Lipschitz (possibly nonsmooth) nonlinearities, without any smallness conditions on the initial data or the running target. These results are motivated by applications in machine learning through deep residual neural networks, which may be fit within our setting. We show that our methodology is applicable to controlled PDEs as well, such as the semilinear wave and heat equation with a globally Lipschitz nonlinearity, once again without any smallness assumptions.

Motivation & Objective

• Address the lack of turnpike results for nonlinear optimal control problems with globally Lipschitz, possibly nonsmooth nonlinearities and large initial data or targets.
• Overcome limitations of traditional methods relying on Pontryagin's Maximum Principle or linearization, which require smoothness and smallness conditions.
• Develop a general framework applicable to both finite-dimensional control-affine systems and infinite-dimensional PDEs, such as semilinear wave and heat equations.
• Enable turnpike analysis in machine learning contexts, particularly deep residual networks (ResNets), where nonlinearities like ReLU are common and often nonsmooth.
• Extend the turnpike property to systems with non-small targets and without requiring $C^2$ regularity or strict dissipativity.

Proposed method

• Construct quasi-turnpike controls using controllability arguments to steer the system toward a steady-state target in finite time.
• Apply a bootstrap argument to iteratively improve the closeness of the trajectory to the turnpike, leveraging the system's Lipschitz continuity.
• Avoid analyzing the optimality system or linearizing the dynamics, thus bypassing the need for differentiability or smallness assumptions.
• Use energy estimates and Gronwall-type inequalities to control the deviation of the state from the target, relying on the global Lipschitz property of the nonlinearity.
• Establish uniform bounds on the control and state trajectories over large time horizons, ensuring exponential convergence to the turnpike.
• Adapt the framework to PDEs by combining semigroup theory, Poincaré and Cauchy-Schwarz inequalities, and variational formulations to derive stability estimates.

Experimental results

Research questions

• RQ1 Can the turnpike property be established for nonlinear optimal control problems with globally Lipschitz, possibly nonsmooth nonlinearities, without smallness assumptions on initial data or the running target?
• RQ2 Is it possible to prove exponential turnpike estimates without relying on linearization or analysis of the optimality system?
• RQ3 Can the proposed method be extended to infinite-dimensional systems such as semilinear wave and heat equations with Lipschitz nonlinearities?
• RQ4 Does the framework apply to machine learning models like ResNets, where the activation functions are typically ReLU or sigmoid—non-differentiable at points?
• RQ5 What are the minimal structural assumptions on the system (e.g., control-affine, globally Lipschitz) for the turnpike property to hold uniformly in the time horizon?

Key findings

• The turnpike property holds for finite-dimensional, control-affine systems with globally Lipschitz nonlinearities, even when the nonlinearity is not $C^1$, and without smallness assumptions on initial data or the target.
• For the semilinear heat and wave equations with globally Lipschitz nonlinearities, the turnpike property is established under the same conditions, without requiring small initial data or target deviations.
• The method constructs a control that steers the system to the target in finite time and maintains it close to the turnpike for most of the time horizon, with exponential convergence rates.
• The proof avoids linearization and optimality system analysis, relying instead on controllability and a bootstrap argument, making it robust to nonsmooth dynamics.
• The framework applies directly to deep residual networks (ResNets), where the activation functions are typically ReLU or sigmoid, showing that the optimal control remains near a steady-state solution for most of the time interval.
• The method fails to extend directly to the cubic heat equation due to the lack of uniform $L^2$ control norm bounds on $u_T$, though the possibility remains open if such bounds can be established.