[Paper Review] Regret-optimal Estimation and Control

Gautam Goel, Babak Hassibi|arXiv (Cornell University)|Jun 22, 2021

Advanced Control Systems Optimization4 citations

TL;DR

This paper introduces regret-optimal estimators and controllers for linear time-varying systems by minimizing the worst-case difference (regret) between online causal policies and a clairvoyant noncausal benchmark. Using operator-theoretic techniques and state-space reformulations, it derives causal filters and controllers that achieve tight, data-dependent regret bounds in terms of disturbance energy, outperforming standard methods like EKF and MPC in numerical experiments.

ABSTRACT

We consider estimation and control in linear time-varying dynamical systems from the perspective of regret minimization. Unlike most prior work in this area, we focus on the problem of designing causal estimators and controllers which compete against a clairvoyant noncausal policy, instead of the best policy selected in hindsight from some fixed parametric class. We show that the regret-optimal estimator and regret-optimal controller can be derived in state-space form using operator-theoretic techniques from robust control and present tight,data-dependent bounds on the regret incurred by our algorithms in terms of the energy of the disturbances. Our results can be viewed as extending traditional robust estimation and control, which focuses on minimizing worst-case cost, to minimizing worst-case regret. We propose regret-optimal analogs of Model-Predictive Control (MPC) and the Extended KalmanFilter (EKF) for systems with nonlinear dynamics and present numerical experiments which show that our regret-optimal algorithms can significantly outperform standard approaches to estimation and control.

Motivation & Objective

• To address the limitation of traditional H₂ and H∞ control, which assume specific disturbance classes and may fail under mismatched disturbances.
• To design adaptive, causal estimators and controllers that minimize regret against a globally optimal, noncausal benchmark with full knowledge of future disturbances.
• To extend the concept of regret minimization beyond fixed parametric policy classes to a more general, nonparametric comparison with the optimal clairvoyant policy.
• To provide tight, data-dependent regret bounds in terms of disturbance energy for both estimation and control problems.

Proposed method

• Formulates regret minimization as a comparison between causal online policies and a clairvoyant noncausal policy that knows the full disturbance sequence in advance.
• Uses operator-theoretic techniques from robust control to derive state-space representations of regret-optimal estimators and controllers.
• Constructs an augmented system with 3n states where the H∞ filter in the new system corresponds to the regret-optimal filter in the original system.
• Applies reductions to extend results to settings with prediction horizons and control delays by reformulating the system dynamics with augmented states.
• Derives regret-optimal Model-Predictive Control (MPC) and Extended Kalman Filter (EKF) analogs for nonlinear systems using the same framework.
• Establishes tight, energy-based regret bounds that scale with the total disturbance energy, independent of the disturbance structure.

Experimental results

Research questions

• RQ1 Can a causal estimator be designed to minimize regret against a smoothed, noncausal estimator that has access to all measurements in advance?
• RQ2 Can a causal controller be designed to minimize regret against a clairvoyant controller that knows all future disturbances?
• RQ3 How can regret-optimal estimation and control be formulated in a way that avoids restrictive parametric assumptions on the policy class?
• RQ4 What are the tight, data-dependent bounds on regret in terms of disturbance energy for linear time-varying systems?
• RQ5 How can the regret-optimal framework be extended to systems with prediction or control delay?

Key findings

• The regret-optimal filter is derived as the H∞ filter in an augmented 3n-state system, providing a drop-in replacement for standard filters like the Kalman and H∞ filters.
• The regret-optimal controller achieves a worst-case regret bounded by a constant times the energy of the disturbances, independent of the disturbance distribution.
• Numerical experiments show that the regret-optimal algorithms significantly outperform standard EKF and MPC in terms of estimation and control cost.
• The framework generalizes to systems with prediction horizons and control delays via state augmentation, preserving regret optimality.
• The regret bounds are tight and data-dependent, scaling linearly with the total disturbance energy, which makes the performance guarantee robust and interpretable.
• The approach extends to nonlinear systems through regret-optimal analogs of EKF and MPC, demonstrating practical relevance beyond linear models.