[Paper Review] Solve the General Constrained Optimal Control Problem with Common Integration Method

Sheng Zhang, Jingyuan Gao|arXiv (Cornell University)|Feb 6, 2018

Spacecraft Dynamics and Control23 references3 citations

TL;DR

This paper proposes a novel numerical approach to solve general constrained optimal control problems (OCPs), including those with intractable path constraints, by transforming them into initial-value problems (IVPs) via a variation-evolving method. It establishes costate-free optimality conditions and derives an integral equation for Karush-Kuhn-Tucker (KKT) multipliers, enabling convergence using standard ODE integration methods.

ABSTRACT

Computation of general state- and/or control-constrained Optimal Control Problems (OCPs) is difficult for various constraints, especially the intractable path constraint. For such problems, the theoretical convergence of numerical algorithms is usually not guaranteed, and the right solution may not be successfully obtained. With the recently proposed Variation Evolving Method (VEM), the evolution equations, which guarantee the convergence towards the optimal solution in theory even for the general constrained OCPs, are derived. In particular, the costate-free optimality conditions are established. Besides the analytic expressions of the costates and the Lagrange multipliers adjoining the terminal constraint, the integral equation that determines the Karush-Kuhn-Tucker (KKT) multiplier variable is also derived. Upon the work in this paper, the general constrained OCPs may be transformed to the Initial-value Problems (IVPs) to be solved, with common Ordinary Differential Equation (ODE) numerical integration methods.

Motivation & Objective

• Address the challenge of solving general optimal control problems with complex state and control constraints, particularly path constraints that are difficult to handle with existing methods.
• Overcome the lack of theoretical convergence guarantees in traditional numerical algorithms for constrained OCPs.
• Develop a systematic framework that ensures convergence to the optimal solution even under general constraint structures.
• Establish costate-free optimality conditions to simplify the solution process and avoid the need for explicit costate computation.
• Derive an integral equation for KKT multipliers to enable numerical solution using standard ODE integration techniques.

Proposed method

• Introduce the Variation Evolving Method (VEM) to derive evolution equations that guarantee convergence to the optimal solution for general constrained OCPs.
• Derive costate-free optimality conditions, eliminating the need to solve for costates explicitly.
• Formulate an integral equation that determines the Karush-Kuhn-Tucker (KKT) multiplier variable associated with constraints.
• Transform the original constrained OCP into an initial-value problem (IVP) using the derived evolution equations and integral constraints.
• Utilize common ordinary differential equation (ODE) numerical integration methods to solve the resulting IVPs.
• Ensure theoretical convergence of the solution process by embedding the KKT conditions directly into the evolution dynamics.

Experimental results

Research questions

• RQ1 Can a general framework be developed to solve constrained optimal control problems with intractable path constraints while guaranteeing convergence?
• RQ2 How can costate-free optimality conditions be derived to simplify the solution of OCPs without requiring explicit costate computation?
• RQ3 What integral equation formulation enables the determination of KKT multipliers in general constrained OCPs?
• RQ4 To what extent can standard ODE integration methods be applied to solve transformed OCPs as initial-value problems?
• RQ5 Can the proposed method ensure theoretical convergence for general OCPs, including those with complex or non-smooth constraints?

Key findings

• The proposed method successfully transforms general constrained optimal control problems into initial-value problems (IVPs), enabling solution via standard ODE integration methods.
• Costate-free optimality conditions are rigorously derived, eliminating the need to solve for costates and simplifying the numerical procedure.
• An integral equation for the KKT multiplier variable is established, which is essential for enforcing constraints in the solution process.
• Theoretical convergence of the solution process is guaranteed even for OCPs with intractable path constraints, addressing a major limitation of existing methods.
• The framework is applicable to a broad class of OCPs, including those with terminal constraints and complex inequality constraints.
• The method is validated through theoretical derivation and is shown to be compatible with common numerical ODE solvers, ensuring practical implementability.