[Paper Review] A Dynamic Programming Approach to Finite-horizon Coherent Quantum LQG Control

Igor G. Vladimirov, Ian R. Petersen|arXiv (Cornell University)|May 9, 2011

Quantum Information and Cryptography6 citations

TL;DR

This paper develops a dynamic programming approach to the finite-horizon coherent quantum LQG control problem by recasting it as a deterministic optimal control problem for a covariance-driven system governed by a differential Lyapunov equation. The key contribution is the derivation of algebraic equations for the optimal controller gain matrices through symplectic invariance and Pontryagin’s minimum principle, establishing a quasi-separation property as a quantum analogue of classical LQG separation.

ABSTRACT

The paper is concerned with the coherent quantum Linear Quadratic Gaussian (CQLQG) control problem for time-varying quantum plants governed by linear quantum stochastic differential equations over a bounded time interval. A controller is sought among quantum linear systems satisfying physical realizability (PR) conditions. The latter describe the dynamic equivalence of the system to an open quantum harmonic oscillator and relate its state-space matrices to the free Hamiltonian, coupling and scattering operators of the oscillator. Using the Hamiltonian parameterization of PR controllers, the CQLQG problem is recast into an optimal control problem for a deterministic system governed by a differential Lyapunov equation. The state of this subsidiary system is the symmetric part of the quantum covariance matrix of the plant-controller state vector. The resulting covariance control problem is treated using dynamic programming and Pontryagin's minimum principle. The associated Hamilton-Jacobi-Bellman equation for the minimum cost function involves Frechet differentiation with respect to matrix-valued variables. The gain matrices of the CQLQG optimal controller are shown to satisfy a quasi-separation property as a weaker quantum counterpart of the filtering/control decomposition of classical LQG controllers.

Motivation & Objective

• To address the finite-horizon coherent quantum LQG (CQLQG) control problem for time-varying quantum plants governed by linear QSDEs.
• To develop a solution framework that respects physical realizability (PR) constraints for quantum controllers.
• To overcome the coupling introduced by PR conditions in quantum LQG control by leveraging Hamiltonian parameterization and symplectic invariance.
• To establish a quantum analogue of the classical LQG separation principle through a quasi-separation property in the optimal controller structure.
• To formulate a split boundary value problem for two nonlinearly coupled Lyapunov ODEs governing the optimal controller gains.

Proposed method

• The CQLQG problem is recast as a deterministic optimal control problem for a subsidiary system whose state is the symmetric part of the quantum covariance matrix of the plant-controller state vector.
• The controller is parameterized via Hamiltonian matrices corresponding to free Hamiltonian, coupling, and scattering operators of an open quantum harmonic oscillator, ensuring physical realizability.
• Dynamic programming and Pontryagin’s minimum principle are applied to derive the Hamilton-Jacobi-Bellman equation involving Fréchet differentiation with respect to matrix-valued variables.
• The costate of the subsidiary system is identified as the observability Gramian of the closed-loop system, enabling a connection between optimal control and system Gramians.
• Symplectic invariance of the minimum cost function is established, reducing the Hamiltonian minimization to two independent quadratic optimization problems.
• The optimal gain matrices are derived in closed form using the Gramians and Hankelian of the closed-loop system, leading to a split boundary value problem for coupled Lyapunov equations.

Experimental results

Research questions

• RQ1 How can the finite-horizon coherent quantum LQG control problem be reformulated as a deterministic optimal control problem under physical realizability constraints?
• RQ2 What is the role of symplectic invariance in simplifying the structure of the optimal quantum controller?
• RQ3 Can a quantum analogue of the classical LQG separation principle be established, and if so, in what form?
• RQ4 How do the optimal controller gain matrices depend on the observability and controllability Gramians of the closed-loop system?
• RQ5 What is the structure of the resulting two-point boundary value problem for the Lyapunov equations governing the optimal controller?

Key findings

• The optimal controller gain matrices are derived in closed form using the observability and controllability Gramians of the closed-loop system, expressed through matrix inverses of special self-adjoint operators.
• The minimum cost function exhibits symplectic invariance, which enables the decoupling of the Hamiltonian minimization into two independent quadratic optimization problems.
• The optimal controller gains satisfy a quasi-separation property, representing a weaker quantum counterpart to the classical LQG filtering/control decomposition.
• The system dynamics preserve the skew-Hamiltonian structure of the Hankelian matrix $ H_t^{22} $ for all $ t < T $, regardless of the choice of the free parameter $ R_t $.
• The resulting controller design leads to a split boundary value problem for two nonlinearly coupled differential Lyapunov equations, with $ R_t $ acting as a free parameter that satisfies the boundary conditions.
• The absence of an equation for the optimal $ R_t $ indicates that the solution structure is constrained by the boundary conditions rather than optimality of $ R_t $, leaving its determination as an open problem.