[Paper Review] Distributed Randomized Control for Demand Dispatch

Ana Bušić, Sean Meyn|arXiv (Cornell University)|Mar 18, 2016

Smart Grid Energy Management8 references16 citations

TL;DR

This paper proposes two novel distributed control designs—Individual Perspective Design (IPD) and System Perspective Design (SPD)—for demand dispatch in power systems using randomized controllers in thermostatically controlled loads. Both methods leverage solutions to a single ordinary differential equation to achieve passivity and stability in aggregate load behavior, enabling flexible loads to provide ancillary services without compromising quality of service.

ABSTRACT

The paper concerns design of control systems for Demand Dispatch to obtain ancillary services to the power grid by harnessing inherent flexibility in many loads. The role of "local intelligence" at the load has been advocated in prior work, randomized local controllers that manifest this intelligence are convenient for loads with a finite number of states. The present work introduces two new design techniques for these randomized controllers: (i) The Individual Perspective Design (IPD) is based on the solution to a one-dimensional family of Markov Decision Processes, whose objective function is formulated from the point of view of a single load. The family of dynamic programming equation appears complex, but it is shown that it is obtained through the solution of a single ordinary differential equation. (ii) The System Perspective Design (SPD) is motivated by a single objective of the grid operator: Passivity of any linearization of the aggregate input-output model. A solution is obtained that can again be computed through the solution of a single ordinary differential equation. Numerical results complement these theoretical results.

Motivation & Objective

• Address the challenge of balancing unpredictable renewable generation by harnessing inherent flexibility in loads for ancillary services.
• Design decentralized, randomized local controllers that allow loads to autonomously adjust power consumption based on grid signals while maintaining quality of service.
• Ensure system-wide stability and passivity of the aggregate load response by formulating control policies from both individual load and grid operator perspectives.
• Develop scalable, distributed control architectures that avoid centralized coordination, enabling large-scale deployment in real-time power systems.

Proposed method

• Formulate the Individual Perspective Design (IPD) as a solution to a one-dimensional family of Markov Decision Processes from the viewpoint of a single load, reducing complex dynamic programming to solving a single ordinary differential equation.
• Develop the System Perspective Design (SPD) based on passivity of the linearized aggregate input-output model, ensuring robustness and stability for the entire system.
• Use a finite-state Markov chain model to represent thermostatically controlled loads (TCLs), with state transitions governed by randomized policies based on temperature thresholds and cumulative distribution functions.
• Implement geometric sampling to model discrete-time renewal processes in TCL dynamics, enabling simulation of realistic load behavior with minimal modeling error.
• Estimate the nominal transition matrix $ Q_0 $ via Monte Carlo simulation or real measurements, ensuring the Markov model accurately reflects the behavior of actual loads.
• Validate designs using Bode plots and linearization analysis across varying operating points $ \zeta $, comparing stability and robustness of IPD and myopic designs.

Experimental results

Research questions

• RQ1 How can decentralized, randomized controllers be designed to allow individual loads to provide ancillary services without degrading quality of service?
• RQ2 What control framework ensures passivity and stability of the aggregate load response when viewed from the grid operator’s perspective?
• RQ3 Can the complex family of dynamic programming equations in the individual load design be reduced to a single ordinary differential equation for tractable computation?
• RQ4 How do the IPD and SPD designs compare in robustness and performance across varying system operating conditions and load dynamics?
• RQ5 To what extent can the input-output behavior of the aggregate load be approximated as linear, and how does this affect system reliability and control design?

Key findings

• The IPD design achieves stable and predictable input-output behavior across a wide range of operating points $ \zeta $, with Bode plots showing consistent linear-like response even for $ |\zeta| > 3 $.
• The SPD design ensures passivity of the linearized aggregate model by solving a single ordinary differential equation, guaranteeing robustness under system-wide linearization.
• The myopic design, while similar to IPD at $ \zeta = 0 $, exhibits highly unpredictable behavior for $ |\zeta| > 3 $, indicating poor robustness and sensitivity to operating point variation.
• Numerical results confirm that both IPD and SPD can be computed efficiently via solution of a single ODE, enabling scalable deployment in large-scale demand dispatch systems.
• The modeling error introduced by geometric sampling is small when $ \gamma $ is not close to unity, validating the use of discrete renewal processes in simulating TCL dynamics.
• The aggregate load behavior can be effectively modeled as a virtual energy storage system, capable of tracking zero-energy signals like $ G_{\text{HP}} $ and $ G_{\text{MP}} $, enabling support for high-frequency grid fluctuations.