[Paper Review] Transform the Non-linear Programming Problem to the Initial-value Problem to Solve

Sheng Zhang, Fei Liao+2 more|arXiv (Cornell University)|Apr 25, 2018

Spacecraft Dynamics and Control34 references3 citations

TL;DR

This paper proposes a dynamic optimization method that transforms nonlinear programming problems with equality and inequality constraints into an initial-value problem (IVP) using a novel Dynamic Optimization Equation (DOE). By leveraging Lyapunov stability theory and matrix pseudo-inverse, the method ensures global convergence to the optimal solution without requiring linear independence of constraints, enabling efficient numerical solution via standard ODE solvers with theoretical guarantees.

ABSTRACT

A dynamic method to solve the Non-linear Programming (NLP) problem with Equality Constraints (ECs) and Inequality Constraints (IECs) is proposed. Inspired by the Lyapunov continuous-time dynamics stability theory in the control field, the optimal solution is analogized to the stable equilibrium point of a finite-dimensional dynamic system and it is solved in an asymptotic manner. Under the premise that the Karush-Kuhn-Tucker (KKT) optimality condition exists, the Dynamic Optimization Equation (DOE), which has the same dimension to that of the optimization parameter vector, is established and its solution will converge to the optimal solution of the NLP globally with a theoretical guarantee. Using the matrix pseudo-inverse, the DOE is valid even without the linearly independent regularity requirement on the nonlinear constraints. In addition, the analytic expressions of the Lagrange multipliers and KKT multipliers, which adjoin the ECs and the IECs respectively during the entire optimization process, are also derived. Via the proposed method, the NLP may be transformed to the Initial-value Problem (IVP) to be solved, with mature Ordinary Differential Equation (ODE) integration methods. Illustrative examples are solved and it is shown that the dynamic method developed may produce the right numerical solutions with high efficiency.

Motivation & Objective

• To develop a dynamic method that reformulates constrained nonlinear programming problems into initial-value problems for numerical solution.
• To ensure global convergence to the optimal solution under the Karush-Kuhn-Tucker (KKT) optimality conditions.
• To eliminate the need for linear independence of nonlinear constraints by employing matrix pseudo-inverse in the formulation.
• To derive analytic expressions for Lagrange and KKT multipliers throughout the optimization process.
• To enable high-efficiency solution using mature ordinary differential equation integration techniques.

Proposed method

• Formulates a Dynamic Optimization Equation (DOE) of the same dimension as the parameter vector, ensuring asymptotic convergence to the optimal solution.
• Applies Lyapunov continuous-time dynamics stability theory to model the optimal solution as a stable equilibrium point of a finite-dimensional system.
• Uses matrix pseudo-inverse to handle non-regular constraint systems, avoiding the need for linear independence assumptions.
• Derives analytic expressions for Lagrange multipliers associated with equality constraints and KKT multipliers for inequality constraints during the entire optimization trajectory.
• Transforms the original nonlinear programming problem into an initial-value problem (IVP) solvable via standard ODE integration methods.
• Employs numerical integration of the DOE to compute the optimal solution with theoretical convergence guarantees.

Experimental results

Research questions

• RQ1 Can a nonlinear programming problem with both equality and inequality constraints be reformulated as an initial-value problem for numerical solution?
• RQ2 How can global convergence to the optimal solution be guaranteed without requiring linear independence of constraints?
• RQ3 What is the role of the Dynamic Optimization Equation (DOE) in ensuring asymptotic stability toward the optimal solution?
• RQ4 How can Lagrange and KKT multipliers be analytically tracked during the entire optimization process?
• RQ5 To what extent does the proposed method improve computational efficiency compared to traditional nonlinear programming solvers?

Key findings

• The proposed method guarantees global convergence to the optimal solution of the nonlinear programming problem under the KKT optimality conditions.
• The Dynamic Optimization Equation (DOE) ensures asymptotic convergence to the optimal solution via Lyapunov stability analysis.
• The use of matrix pseudo-inverse allows the method to remain valid even when constraints are not linearly independent.
• Analytic expressions for Lagrange and KKT multipliers are derived and maintained throughout the optimization process.
• Illustrative examples demonstrate that the method produces accurate numerical solutions with high efficiency using standard ODE solvers.
• The transformation to an initial-value problem enables the use of well-established and robust ODE integration techniques for solving complex nonlinear programs.