[Paper Review] A monotonic method for solving nonlinear optimal control problems

Julien Salomon, Gabriel Turinici|arXiv (Cornell University)|Jun 18, 2009

Advanced Optimization Algorithms Research48 references3 citations

TL;DR

This paper presents a unified monotonic algorithm for solving nonlinear optimal control problems with concave dependence on the state, proving the existence of solutions to a nonlinear evolution equation that ensures monotonic decrease in the cost functional. The method generalizes prior approaches in quantum and parabolic control, offering a constructive, well-posed procedure that outperforms gradient methods in convergence speed for non-convex settings.

ABSTRACT

Initially introduced in the framework of quantum control, the so-called "monotonic algorithms" have demonstrated excellent numerical performance when dealing with bilinear optimal control problems. This paper presents a unified formulation that can be applied to more nonlinear settings compatible with the hypothesis detailed below. In this framework, we show that the well-posedness of the general algorithm is related to a nonlinear evolution equation. We prove the existence of the solution to this equation and give important properties of the optimal control functional. Finally we show how the algorithm works for selected models from the literature and compare it with the gradient algorithm.

Motivation & Objective

• To unify disparate monotonic algorithms used in quantum and parabolic control into a single theoretical framework.
• To prove the well-posedness of the monotonic algorithm by establishing existence of solutions to a nonlinear evolution equation.
• To extend the applicability of monotonic methods beyond bilinear and quadratic settings to general nonlinear controls with concave state dependence.
• To provide a constructive procedure for computing each iteration’s control update, ensuring monotonic decrease in the cost functional.
• To demonstrate the method’s efficiency through numerical comparisons with gradient-based algorithms on representative models.

Proposed method

• The algorithm constructs a sequence of controls $v^k$ such that $J(v^{k+1}) \leq J(v^k)$, ensuring monotonic cost reduction by design.
• Well-posedness of the algorithm is linked to solving a nonlinear evolution equation derived from the optimality condition.
• The existence of a solution to this evolution equation is rigorously proven under hypotheses of concave dependence of the cost functional on the state.
• The method uses a variational formulation involving the Fréchet derivative and adjoint state $Y$, leading to a nonlinear equation in the control increment $v'$.
• A constructive procedure is provided to compute $v^{k+1}$ from $v^k$ via solution of the nonlinear equation $\Delta(v', v; t, X, Y) = -\theta(v' - v)$.
• Numerical approximations are based on the constructive proof, enabling implementation on models with arbitrary nonlinear $A(t,v)$ as long as the concavity condition holds.

Experimental results

Research questions

• RQ1 Can a unified theoretical framework be established for monotonic algorithms across diverse nonlinear optimal control problems?
• RQ2 Under what conditions does the monotonic algorithm remain well-posed when $A(t,v)$ is nonlinear in $v$ and $J$ is concave in the state?
• RQ3 Can the existence of a control update $v^{k+1}$ that ensures $J(v^{k+1}) \leq J(v^k)$ be proven constructively in general nonlinear settings?
• RQ4 How does the monotonic algorithm compare in convergence speed and robustness to gradient-based methods in non-convex control problems?
• RQ5 What are the structural properties of the optimal control functional under the proposed framework?

Key findings

• The monotonic algorithm is proven to be well-posed for a broad class of nonlinear optimal control problems with concave dependence of the cost functional on the state.
• A solution to the nonlinear evolution equation governing the control update exists under the stated hypotheses, enabling a constructive numerical procedure.
• The algorithm ensures monotonic decrease in the cost functional at each iteration without requiring line search or additional computational effort.
• Numerical results show that the monotonic algorithm converges asymptotically faster than the gradient method, especially in non-convex settings.
• In the example of controlling a nonlinear Bose-Einstein condensate, the monotonic method achieves faster convergence than the gradient method despite slower initial progress.
• The method successfully handles vectorial controls and higher-order nonlinearities, such as cubic dependence in the control, as demonstrated in the molecular orientation control example.