[Paper Review] Measurement-Feedback Control with Optimal Data-Dependent Regret

Gautam Goel, Babak Hassibi|arXiv (Cornell University)|Sep 14, 2022

Advanced Bandit Algorithms Research4 citations

TL;DR

This paper introduces regret-optimal measurement-feedback controllers for linear systems, achieving optimal data-dependent regret by reducing the problem to H∞-optimal control in a synthetic system. It derives two controllers: one with optimal dependence on joint disturbance energy and another on driving disturbance pathlength and measurement disturbance energy, proving that bounded competitive ratio is unattainable in general.

ABSTRACT

Inspired by online learning, data-dependent regret has recently been proposed as a criterion for controller design. In the regret-optimal control paradigm, causal controllers are designed to minimize regret against a hypothetical optimal noncausal controller, which selects the globally cost-minimizing sequence of control actions given noncausal access to the disturbance sequence. We extend regret-optimal control to the more challenging measurement-feedback setting, where the online controller must compete against the optimal noncausal controller without directly observing the state or the driving disturbance. We show that no measurement-feedback controller can have bounded competitive ratio or regret which is bounded by the pathlength of the measurement disturbance. We do derive, however, a controller whose regret has optimal dependence on the joint energy of the driving and measurement disturbances, and another controller whose regret has optimal dependence on the pathlength of the driving disturbance and the energy of the measurement disturbance. The key technique we introduce is a reduction from regret-optimal measurement-feedback control to $H_{\infty}$-optimal measurement-feedback control in a synthetic system. We present numerical simulations which illustrate the efficacy of our proposed control algorithms.

Motivation & Objective

• Address the challenge of designing causal controllers that minimize regret against a noncausal optimal controller in measurement-feedback settings.
• Overcome the limitation that bounded competitive ratio or pathlength-bounded regret is unattainable in general linear systems under measurement feedback.
• Develop controllers with optimal regret dependence on disturbance complexity measures: joint energy and mixed pathlength-energy.
• Establish a reduction framework from regret-optimal measurement-feedback control to H∞-optimal control in a synthetic system to enable tractable design.

Proposed method

• Formulate the regret-optimal measurement-feedback control problem as a minimization of regret against a clairvoyant noncausal controller.
• Introduce a synthetic system where the original measurement-feedback problem is transformed into an H∞-optimal control problem.
• Use a state-space transformation to embed the original system dynamics and disturbance models into the synthetic system with augmented state and control inputs.
• Apply the H∞-optimal control framework to the synthetic system to derive controllers with provably optimal regret bounds.
• Derive two distinct controllers: one minimizing regret with optimal dependence on the joint energy of driving and measurement disturbances, and another with optimal pathlength-energy trade-off.
• Leverage the separation principle and Riccati equation solutions to ensure stability and optimality of the derived controllers.

Experimental results

Research questions

• RQ1 Can measurement-feedback controllers achieve bounded competitive ratio or regret bounded by the pathlength of measurement disturbances?
• RQ2 What is the optimal regret dependence achievable for measurement-feedback controllers under different disturbance complexity measures?
• RQ3 Can regret-optimal measurement-feedback control be reduced to an H∞-optimal control problem in a synthetic system?
• RQ4 How do the proposed controllers compare in performance to standard H₂ and H∞ controllers across diverse disturbance types?
• RQ5 What are the fundamental limitations of regret minimization in measurement-feedback control settings?

Key findings

• No measurement-feedback controller can achieve bounded competitive ratio or regret bounded by the pathlength of the measurement disturbance.
• The proposed controller achieves regret with optimal dependence on the joint energy of driving and measurement disturbances, matching theoretical lower bounds.
• A second controller achieves regret with optimal dependence on the pathlength of the driving disturbance and the energy of the measurement disturbance.
• Numerical simulations on a double integrator system show that the energy-optimal controller outperforms others under i.i.d. Gaussian disturbances.
• The pathlength-optimal controller closely tracks the noncausal benchmark under Gaussian random walk disturbances, where pathlength is low relative to energy.
• In adversarial impulse disturbances, the H∞-optimal and energy-optimal controllers outperform H₂ and pathlength-optimal controllers, validating robustness.