[Paper Review] Optimization Algorithms as Robust Feedback Controllers

Adrian Hauswirth, He, Zhiyu+3 more|arXiv (Cornell University)|Mar 21, 2021

Extremum Seeking Control Systems32 citations

TL;DR

A survey framing optimization algorithms as dynamical systems and feedback controllers, focusing on closed-loop stability and constraint handling in data-driven settings, with applications to power grids and communications.

ABSTRACT

Mathematical optimization is one of the cornerstones of modern engineering research and practice. Yet, throughout all application domains, mathematical optimization is, for the most part, considered to be a numerical discipline. Optimization problems are formulated to be solved numerically with specific algorithms running on microprocessors. An emerging alternative is to view optimization algorithms as dynamical systems. Besides being insightful in itself, this perspective liberates optimization methods from specific numerical and algorithmic aspects and opens up new possibilities to endow complex real-world systems with sophisticated self-optimizing behavior. Towards this goal, it is necessary to understand how numerical optimization algorithms can be converted into feedback controllers to enable robust "closed-loop optimization". In this article, we focus on recent control designs under the name of "feedback-based optimization" which implement optimization algorithms directly in closed loop with physical systems. In addition to a brief overview of selected continuous-time dynamical systems for optimization, our particular emphasis in this survey lies on closed-loop stability as well as the robust enforcement of physical and operational constraints in closed-loop implementations. To bypass accessing partial model information of physical systems, we further elaborate on fully data-driven and model-free operations. We highlight an emerging application in autonomous reserve dispatch in power systems, where the theory has transitioned to practice by now. We also provide short expository reviews of pioneering applications in communication networks and electricity grids, as well as related research streams, including extremum seeking and pertinent methods from model predictive and process control, to facilitate high-level comparisons with the main topic of this survey.

Motivation & Objective

• Motivate and formalize the view of optimization algorithms as dynamical systems that can be closed in with physical plants.
• Explore how feedback-based optimization achieves robustness to model inaccuracies and time-varying parameters.
• Examine mechanisms for enforcing hard and soft constraints in closed-loop optimization.
• Discuss data-driven and model-free operation for self-optimizing autonomous systems.
• Highlight applications in autonomous reserve dispatch and related domains to illustrate theory in practice.

Proposed method

• Review continuous-time optimization dynamics such as gradient flows, projected gradient flows, and primal-dual saddle-point flows as dynamical systems.
• Discuss closed-loop interconnections with physical plants and the role of singular perturbation and stability analyses.
• Present mechanisms for constraint enforcement, including projection, anti-windup, dualization, and approximate projections.
• Outline data-driven and model-free strategies, including online sensitivity learning and derivative-free optimization.
• Provide a case study on real-time optimal power flow and automatic redispatch in electricity grids.
• Compare with related approaches like extremum seeking, MPC with incomplete optimization, and modifier adaptation.

Experimental results

Research questions

• RQ1 How can optimization algorithms be interpreted as dynamical systems and integrated in closed-loop with physical plants?
• RQ2 What are the stability and robustness conditions for closed-loop optimization under data uncertainty and time-varying parameters?
• RQ3 How can hard and soft constraints be effectively enforced in closed-loop optimization without full model dependence?
• RQ4 What data-driven or model-free techniques enable autonomous, measurement-driven optimization in real time?
• RQ5 How do these theories apply to real-world applications such as autonomous reserve dispatch in power systems?

Key findings

• Optimization dynamics can be interconnected with plants to drive outputs toward optimizer-based equilibria.
• Stability results link to singular perturbations and saddle-point dynamics, informing design of stable closed-loop controllers.
• Projected gradient and saddle-point flows provide mechanisms to handle inequality constraints and equalities in closed-loop settings.
• Anti-windup and dualization strategies enable constraint enforcement in the presence of actuation limits and saturations.
• Data-driven and online sensitivity learning enable model-free operation while preserving convergence toward optimum.
• A real-time application in electricity systems demonstrates closed-loop optimization for autonomous redispatch and real-time power flow.