[Paper Review] Data-Driven Linear Quadratic Optimization for Controller Synthesis with Structural Constraints

Jun Ma, Zilong Cheng+4 more|arXiv (Cornell University)|Dec 8, 2019

Advanced Control Systems Optimization4 citations

TL;DR

This paper proposes a data-driven gradient projection method for solving linear quadratic regulator (LQR) problems with structural constraints on the controller gain matrix, such as fixed zero entries. By computing gradients from system data and projecting them onto constraint hyperplanes, the algorithm iteratively updates the gain matrix to minimize cost while preserving structure, achieving lower cost than model-based methods under model uncertainty.

ABSTRACT

For various typical cases and situations where the formulation results in an optimal control problem, the Linear Quadratic Regulator (LQR) approach and its variants continue to be highly attractive. In certain scenarios, it can happen that some prescribed structural constraints on the gain matrix would arise. Consequently then, the Algebraic Riccati Equation (ARE) is no longer applicable in a straightforward way to obtain the optimal solution. This work presents a rather effective alternative optimization approach based on gradient projection. The utilized gradient is obtained through a data-driven methodology, and then projected onto applicable constrained hyperplanes. Essentially, this projection gradient determines a direction of progression and computation for the gain matrix update with a decreasing functional cost; and then the gain matrix is further refined in an iterative framework. With this formulation, a data-driven optimization algorithm is summarized for controller synthesis with structural constraints. This data-driven approach has the key advantage that it avoids the necessity of precise modeling which is always required in the classical model-based counterpart; and thus the approach can additionally accommodate various model uncertainties. Illustrative examples are also provided in the work to validate the theoretical results.

Motivation & Objective

• Address the challenge of solving constrained LQR problems where structural constraints (e.g., zero entries) in the controller gain matrix prevent direct use of the Algebraic Riccati Equation.
• Develop a model-free optimization framework that avoids reliance on precise system models, which is critical in the presence of model uncertainty.
• Enable effective controller synthesis for decentralized or structurally restricted control architectures using only system input-output data.
• Ensure convergence and monotonic cost reduction during optimization through a projected gradient descent approach.
• Demonstrate the superiority of the data-driven method over traditional model-based approaches in scenarios with significant model inaccuracies.

Proposed method

• Derive the gradient of the LQR cost function with respect to the controller gain matrix using data from system trajectories, avoiding explicit system model knowledge.
• Formulate structural constraints (e.g., fixed zero entries) as linear equality constraints on the gain matrix, represented via matrices $ G_i $ and $ H_i $ such that $ G_i K H_i = 0 $.
• Apply a gradient projection method to project the descent direction onto the feasible set defined by the structural constraints, ensuring constraint preservation during updates.
• Iteratively update the controller gain matrix using the projected gradient to minimize the cost function while maintaining structural sparsity.
• Use a line search or fixed step size to ensure convergence and cost reduction at each iteration.
• Implement the algorithm in a data-driven framework where only input and state trajectories are required, not system dynamics matrices.

Experimental results

Research questions

• RQ1 Can a data-driven optimization method effectively solve LQR problems with structural constraints on the controller gain when the system model is unknown or uncertain?
• RQ2 How does the performance of a data-driven gradient projection method compare to a model-based counterpart under model mismatch or uncertainty?
• RQ3 Does the proposed method preserve structural constraints (e.g., sparsity or decentralized structure) while ensuring monotonic cost reduction and convergence?
• RQ4 What is the impact of data quality and system dimensionality on the scalability and convergence of the data-driven LQR optimization?
• RQ5 Can the method be generalized to handle multiple types of structural constraints beyond fixed zero entries in the gain matrix?

Key findings

• The proposed data-driven method achieved a cost of $ 8.2534 imes 10^2 $ after 10 iterations, significantly lower than the model-based method’s cost of $ 1.2060 imes 10^3 $, demonstrating superior performance under model uncertainty.
• The norm of the constraint violation was reduced to $ 5.6196 imes 10^{-5} $, indicating tight satisfaction of structural constraints during optimization.
• The optimal gain matrix $ K^* = egin{bmatrix}-57.8053 & -18.8322 & 0 & 0 \ 0 & 0 & -38.0592 & -7.3845 ight] $ was successfully computed using only data, without system model knowledge.
• System response simulations (Fig. 7) confirmed stable and well-performing closed-loop dynamics with the data-driven controller under the initial condition $ x_0 = [1 ext{ }0 ext{ }1 ext{ }0]^T $.
• The model-based method, while achieving lower cost in nominal conditions, failed to maintain performance on the perturbed system due to model mismatch, highlighting the robustness of the data-driven approach.
• The method successfully preserved the required zero-structure in the gain matrix throughout iterations, confirming the effectiveness of the gradient projection mechanism.