[Paper Review] Energy Concerns with HPC Systems and Applications

R. Kanyou Nana, Claude Tadonki+2 more|arXiv (Cornell University)|Aug 31, 2023

Parallel Computing and Optimization TechniquesComputer Science3 citations

TL;DR

This survey paper examines energy efficiency challenges in high-performance computing (HPC) and embedded systems, analyzing hardware architectures, energy metrics, cooling systems, and optimization techniques. It highlights that energy efficiency is critical for reducing operational costs and carbon emissions, with key findings showing that specialized accelerators like TPUs and efficient microcontrollers can improve performance/watt by 2–5x and enable 20x better energy efficiency over CPUs in AI inference workloads.

ABSTRACT

For various reasons including those related to climate changes, {\em energy} has become a critical concern in all relevant activities and technical designs. For the specific case of computer activities, the problem is exacerbated with the emergence and pervasiveness of the so called {\em intelligent devices}. From the application side, we point out the special topic of {\em Artificial Intelligence}, who clearly needs an efficient computing support in order to succeed in its purpose of being a {\em ubiquitous assistant}. There are mainly two contexts where {\em energy} is one of the top priority concerns: {\em embedded computing} and {\em supercomputing}. For the former, power consumption is critical because the amount of energy that is available for the devices is limited. For the latter, the heat dissipated is a serious source of failure and the financial cost related to energy is likely to be a significant part of the maintenance budget. On a single computer, the problem is commonly considered through the electrical power consumption. This paper, written in the form of a survey, we depict the landscape of energy concerns in computer activities, both from the hardware and the software standpoints.

Motivation & Objective

• Address the growing energy cost and carbon footprint associated with high-performance computing (HPC) and embedded systems.
• Examine energy efficiency as a critical constraint in both HPC (due to cooling and operational costs) and embedded systems (due to limited battery power).
• Investigate the role of energy-aware design across hardware, software, and system-level optimizations.
• Evaluate the impact of AI workloads on energy consumption and explore optimization strategies to reduce their energy footprint.
• Provide a comprehensive overview of energy metrics, profiling tools, cooling technologies, and optimization techniques for diverse computing platforms.

Proposed method

• Survey of existing literature on energy metrics, power measurement, and energy profiling tools across HPC and embedded systems.
• Categorization and analysis of energy-efficient hardware: GPUs, TPUs, FPGAs, microcontrollers (e.g., Arduino, Raspberry Pi, Coral Dev Board), and SoCs.
• Evaluation of energy management tools for CPUs, GPUs, and microcontrollers, including runtime monitoring and dynamic voltage/frequency scaling.
• Review of cooling technologies, especially liquid cooling trends, for HPC systems to manage heat dissipation and improve efficiency.
• Systematic analysis of static, dynamic, and hybrid energy optimization techniques across software stacks and system layers.
• Investigation of AI-specific energy optimization: quantization, pruning, model compression, neural architecture search, and knowledge distillation, with hardware selection (e.g., TPUs, A100 GPUs) as a key enabler.

Figure 1. Taxonomy of power/energy management solutions with related approaches and tools.

Experimental results

Research questions

• RQ1 How do energy consumption and carbon footprint metrics correlate in modern HPC and embedded systems?
• RQ2 What are the most effective hardware accelerators and microcontroller platforms for minimizing energy use in AI inference and HPC workloads?
• RQ3 How do dynamic and static energy optimization techniques compare in terms of performance and energy savings across different system layers?
• RQ4 To what extent can model compression and knowledge distillation reduce the energy cost of AI inference without sacrificing accuracy?
• RQ5 What role do cooling systems and system-level policies play in achieving sustainable HPC and embedded computing?

Key findings

• The Frontier exascale system consumes 21.1 MW for 1.102 EFlop/s, Fugaku 29.9 MW for 442 PFlop/s, and Lumi 2.94 MW for 151.9 PFlop/s, highlighting the growing energy cost of HPC systems.
• Coral Dev Board microcontrollers outperform Intel Skylake CPUs by over 20x in both time-to-solution and energy efficiency for AI inference workloads.
• Using TPUs or A100 GPUs instead of general-purpose CPUs improves performance/watt by factors of 2 to 5 for machine learning training.
• Quantization and knowledge distillation can reduce model size by up to 40% and 60% respectively while preserving 97% of BERT’s language understanding capability.
• Static power consumption significantly impacts data center energy footprint, necessitating effective idle-state power management.
• Liquid cooling and chip-level thermal management are emerging as essential solutions for scalable and efficient HPC cooling, with quantum cooling being explored for future systems.

Figure 2. READEX working diagram