[Paper Review] An Energy-Aware Online Learning Framework for Resource Management in Heterogeneous Platforms

Sumit K. Mandal, Ganapati Bhat+3 more|arXiv (Cornell University)|Mar 20, 2020

Parallel Computing and Optimization Techniques4 citations

TL;DR

This paper proposes an online imitation learning (Online-IL) framework that dynamically adapts resource management policies in heterogeneous mobile platforms to minimize energy consumption while maintaining performance. By leveraging offline-trained policies and real-time power-performance models for strong supervision, Online-IL efficiently learns optimal voltage/frequency and core configuration policies for unseen applications with minimal runtime overhead and significant energy and performance gains.

ABSTRACT

Mobile platforms must satisfy the contradictory requirements of fast response time and minimum energy consumption as a function of dynamically changing applications. To address this need, system-on-chips (SoC) that are at the heart of these devices provide a variety of control knobs, such as the number of active cores and their voltage/frequency levels. Controlling these knobs optimally at runtime is challenging for two reasons. First, the large configuration space prohibits exhaustive solutions. Second, control policies designed offline are at best sub-optimal since many potential new applications are unknown at design-time. We address these challenges by proposing an online imitation learning approach. Our key idea is to construct an offline policy and adapt it online to new applications to optimize a given metric (e.g., energy). The proposed methodology leverages the supervision enabled by power-performance models learned at runtime. We demonstrate its effectiveness on a commercial mobile platform with 16 diverse benchmarks. Our approach successfully adapts the control policy to an unknown application after executing less than 25% of its instructions.

Motivation & Objective

• To address the challenge of dynamic, energy-efficient resource management in heterogeneous mobile SoCs with unknown, evolving workloads.
• To overcome the limitations of offline policies and inefficient reinforcement learning by enabling fast adaptation to new applications at runtime.
• To reduce energy consumption and improve response time in mobile platforms without relying on performance-hungry trial-and-error learning.
• To minimize runtime and storage overhead while maintaining high accuracy in policy adaptation for new workloads.
• To demonstrate the effectiveness of online imitation learning using real-world benchmarks on a commercial mobile platform.

Proposed method

• The framework uses an offline policy as a bootstrap for online policy learning, initialized with known application behaviors.
• At runtime, workload metadata (e.g., instructions, memory accesses) is collected to guide configuration evaluation.
• Analytical power-performance models provide fast, accurate supervision to evaluate candidate configurations without full execution.
• A neural network policy is updated via backpropagation using this model-supervised feedback to refine control decisions.
• The system performs online adaptation in epochs, with periodic model updates and policy refinement based on new training examples.
• Runtime overhead is minimized through efficient sampling and infrequent backpropagation updates (every 100 epochs).

Experimental results

Research questions

• RQ1 Can an online imitation learning approach effectively adapt resource management policies to unseen mobile applications with minimal runtime overhead?
• RQ2 How does Online-IL compare to traditional governors (e.g., powersave, interactive) in terms of energy efficiency and execution time?
• RQ3 To what extent can Online-IL outperform offline learning techniques like DyPO for unknown workloads?
• RQ4 How does the integration of real-time power-performance models improve learning efficiency compared to standard reinforcement learning?
• RQ5 What is the cumulative runtime and storage overhead of Online-IL in a real-world mobile platform?

Key findings

• The proposed Online-IL framework reduces energy consumption by an average of 6.39% and improves execution time by 15.24% over the DyPO offline method for unseen applications.
• For the Kmeans-BML concurrent workload, Online-IL reduces energy by 5% and execution time by 25% compared to the powersave governor.
• Online-IL achieves 24% faster execution and over 10% lower energy use compared to the powersave governor on average across benchmarks.
• The framework incurs only 0.06–0.12% runtime overhead for policy evaluation and less than 1% for backpropagation updates, with 18KB storage overhead for training buffers.
• Compared to reinforcement learning, Online-IL requires only one backpropagation step for optimal policy learning on the Spectral benchmark, while RL fails to converge after eight steps.
• The method successfully adapts to unknown applications after executing less than 25% of their instructions, demonstrating rapid convergence.