[Paper Review] Analysis of the Optimization Landscape of Linear Quadratic Gaussian (LQG) Control

Yang Zheng, Yujie Tang+1 more|arXiv (Cornell University)|Feb 8, 2021

Advanced Control Systems Optimization18 citations

TL;DR

This paper analyzes the optimization landscape of Linear Quadratic Gaussian (LQG) control from a modern non-convex optimization perspective, focusing on the connectivity of stabilizing controllers and the structure of stationary points. It proves that the set of stabilizing controllers has at most two path-connected components, which are diffeomorphic via similarity transformations, and shows that all minimal stationary points are globally optimal, while non-minimal ones are strictly suboptimal and abundant.

ABSTRACT

This paper revisits the classical Linear Quadratic Gaussian (LQG) control from a modern optimization perspective. We analyze two aspects of the optimization landscape of the LQG problem: 1) connectivity of the set of stabilizing controllers $\mathcal{C}_n$; and 2) structure of stationary points. It is known that similarity transformations do not change the input-output behavior of a dynamical controller or LQG cost. This inherent symmetry by similarity transformations makes the landscape of LQG very rich. We show that 1) the set of stabilizing controllers $\mathcal{C}_n$ has at most two path-connected components and they are diffeomorphic under a mapping defined by a similarity transformation; 2) there might exist many \emph{strictly suboptimal stationary points} of the LQG cost function over $\mathcal{C}_n$ and these stationary points are always \emph{non-minimal}; 3) all \emph{minimal} stationary points are globally optimal and they are identical up to a similarity transformation. These results shed some light on the performance analysis of direct policy gradient methods for solving the LQG problem.

Motivation & Objective

• To understand the geometric and analytical properties of the LQG optimization landscape, particularly the connectivity of the feasible set of stabilizing controllers.
• To investigate the role of similarity transformations in shaping the non-convex landscape of LQG control.
• To characterize the structure of stationary points, especially distinguishing between minimal and non-minimal controllers.
• To provide theoretical foundations for the convergence and performance of gradient-based methods in LQG control.
• To extend insights from LQR optimization to the more complex LQG setting with partial observability and dynamical controllers.

Proposed method

• Analyzes the set of strictly proper stabilizing dynamical controllers, denoted $\mathcal{C}_n$, using differential topology to study path-connected components.
• Applies similarity transformations to establish diffeomorphism between connected components, preserving input-output behavior and LQG cost.
• Uses the notion of minimal controllers (controllable and observable) to classify stationary points and exploit symmetry in the optimization landscape.
• Derives conditions under which $\mathcal{C}_n$ is connected, showing that open-loop stable systems always yield a connected feasible set.
• Employs Riccati equations and matrix factorizations (e.g., $X_{11} - X_{12}X_{22}^{-1}X_{12}^T$) to verify stationarity and optimality conditions.
• Constructs explicit families of non-minimal stationary points for open-loop stable systems and derives a Hessian criterion for their non-optimality.

Experimental results

Research questions

• RQ1 How many path-connected components can the set of stabilizing controllers $\mathcal{C}_n$ have?
• RQ2 Are the connected components of $\mathcal{C}_n$ diffeomorphic under similarity transformations?
• RQ3 What is the structural relationship between minimal controllers and global optimality in LQG?
• RQ4 Can strictly suboptimal stationary points exist, and if so, are they always non-minimal?
• RQ5 Under what conditions is the set $\mathcal{C}_n$ guaranteed to be connected?

Key findings

• The set of stabilizing controllers $\mathcal{C}_n$ has at most two path-connected components, and these components are diffeomorphic via a similarity transformation.
• All minimal stationary points of the LQG cost function are globally optimal and are equivalent up to a similarity transformation.
• There exist infinitely many strictly suboptimal stationary points, and all such points are non-minimal.
• For open-loop stable systems, the set $\mathcal{C}_n$ is always connected, and a sufficient condition for connectivity is derived.
• A criterion is established to check whether the Hessian at a non-minimal stationary point is indefinite, confirming suboptimality.
• The LQG cost function exhibits rich symmetry under similarity transformations, which underpins the landscape structure and explains the prevalence of suboptimal stationary points.