[Paper Review] Deep Reinforcement Learning for Optimal Control of Space Heating

Ádám Nagy, Hussain Kazmi+2 more|arXiv (Cornell University)|May 10, 2018

Building Energy and Comfort Optimization21 references53 citations

TL;DR

A novel deep reinforcement learning algorithm for space heating control that is computationally efficient, benchmarked against other techniques, and improves rule-based control by 5–10% under various price signals.

ABSTRACT

Classical methods to control heating systems are often marred by suboptimal performance, inability to adapt to dynamic conditions and unreasonable assumptions e.g. existence of building models. This paper presents a novel deep reinforcement learning algorithm which can control space heating in buildings in a computationally efficient manner, and benchmarks it against other known techniques. The proposed algorithm outperforms rule based control by between 5-10% in a simulation environment for a number of price signals. We conclude that, while not optimal, the proposed algorithm offers additional practical advantages such as faster computation times and increased robustness to non-stationarities in building dynamics.

Motivation & Objective

• Develop a DRL-based control policy for space heating that does not rely on accurate building models.
• Benchmark the DRL controller against traditional rule-based and other control techniques.
• Assess computational efficiency and robustness to non-stationarities in building dynamics.
• Evaluate performance across multiple price signal scenarios.

Proposed method

• Propose a novel deep reinforcement learning algorithm tailored for space heating control.
• Benchmark the proposed DRL method against rule-based control and other techniques in simulations.
• Use simulation environments with various price signals to evaluate performance.
• Compare computational efficiency and robustness to non-stationarities in building dynamics.
• Analyze outcomes in terms of energy cost reduction and adaptability to dynamic conditions.

Experimental results

Research questions

• RQ1 Can a DRL-based controller achieve lower energy costs for space heating compared to rule-based control across multiple price signals?
• RQ2 Is the proposed DRL controller computationally efficient and robust to non-stationary building dynamics?
• RQ3 How does the DRL method perform relative to existing control techniques under dynamic price signals?
• RQ4 What practical advantages does the DRL approach offer beyond optimality (e.g., robustness, adaptation)?

Key findings

• The proposed DRL algorithm outperforms rule-based control by between 5-10% in simulations across several price signals.
• The algorithm is computationally efficient compared to alternatives.
• The DRL approach shows increased robustness to non-stationarities in building dynamics.
• Benchmarking against other known techniques demonstrates competitive performance.