[Paper Review] Target Tracking for Contextual Bandits: Application to Demand Side Management

Margaux Brégère, Pierre Gaillard+2 more|arXiv (Cornell University)|Jan 28, 2019

Advanced Bandit Algorithms Research4 citations

TL;DR

This paper proposes a contextual bandit framework for demand-side management by dynamically adjusting electricity prices to track a target mean consumption, using linear and generalized additive models to predict consumption based on contextual factors like temperature. It achieves a T^{2/3} regret bound (up to logarithmic terms), demonstrating effective demand tracking in real-world simulations with UK Power Networks data.

ABSTRACT

We propose a contextual-bandit approach for demand side management by offering price incentives. More precisely, a target mean consumption is set at each round and the mean consumption is modeled as a complex function of the distribution of prices sent and of some contextual variables such as the temperature, weather, and so on. The performance of our strategies is measured in quadratic losses through a regret criterion. We offer $T^{2/3}$ upper bounds on this regret (up to poly-logarithmic terms)---and even faster rates under stronger assumptions---for strategies inspired by standard strategies for contextual bandits (like LinUCB, see Li et al., 2010). Simulations on a real data set gathered by UK Power Networks, in which price incentives were offered, show that our strategies are effective and may indeed manage demand response by suitably picking the price levels.

Motivation & Objective

• To develop a dynamic pricing strategy that tracks a time-varying target mean electricity consumption using contextual bandit algorithms.
• To model electricity consumption as a function of price levels and contextual variables such as temperature and weather.
• To minimize quadratic loss in tracking the target consumption through regret minimization.
• To evaluate the method on real-world data from UK Power Networks to demonstrate practical effectiveness.

Proposed method

• The mean consumption is modeled as a function of price tariffs and contextual variables using linear or generalized additive models.
• A contextual bandit algorithm is employed where the provider selects price allocations based on context to minimize tracking error.
• The algorithm uses a confidence bound approach inspired by LinUCB, with a regret analysis based on the inverse of the covariance matrix of features.
• The loss function is defined as the squared deviation between observed mean consumption and the target, and regret is bounded using concentration inequalities.
• The method incorporates a noise model with empirical covariance matrix estimated from historical residuals.
• Theoretical bounds are derived using matrix concentration and logarithmic trace inequalities to control the growth of uncertainty.

Experimental results

Research questions

• RQ1 Can contextual bandit algorithms effectively track a moving target in electricity demand by adjusting price incentives?
• RQ2 What regret bounds can be achieved in this target-tracking setting for demand-side management?
• RQ3 How does the performance of the proposed algorithm compare to baseline strategies on real-world data?
• RQ4 What is the impact of contextual variables such as temperature and time-of-day on consumption prediction accuracy?
• RQ5 Under what conditions can faster than T^{2/3} convergence rates be achieved?

Key findings

• The proposed algorithm achieves a T^{2/3} regret bound with high probability, up to poly-logarithmic factors, under general assumptions.
• Under stronger assumptions, such as bounded noise and well-conditioned feature matrices, faster convergence rates are possible.
• Simulations on real UK Power Networks data show that the method effectively tracks target consumption levels using price incentives.
• The empirical covariance matrix of residuals is significantly non-diagonal, indicating correlated noise across price tariffs, which the model accounts for.
• The method outperforms static pricing strategies by dynamically adapting to changing contexts and consumption targets.
• The theoretical analysis confirms that the algorithm balances exploration and exploitation through confidence bounds on model parameters.