[Paper Review] The Impact of GPU DVFS on the Energy and Performance of Deep Learning: an Empirical Study

Zhenheng Tang, Yuxin Wang+2 more|arXiv (Cornell University)|May 27, 2019

Advanced Neural Network Applications45 references5 citations

TL;DR

This empirical study investigates GPU Dynamic Voltage and Frequency Scaling (DVFS) to optimize energy efficiency in deep learning workloads. By testing multiple GPU architectures, DNN configurations, and convolution algorithms, the authors identify optimal core frequencies that reduce energy consumption by up to 23.1% during training and 26.4% during inference, without sacrificing performance.

ABSTRACT

Over the past years, great progress has been made in improving the computing power of general-purpose graphics processing units (GPGPUs), which facilitates the prosperity of deep neural networks (DNNs) in multiple fields like computer vision and natural language processing. A typical DNN training process repeatedly updates tens of millions of parameters, which not only requires huge computing resources but also consumes significant energy. In order to train DNNs in a more energy-efficient way, we empirically investigate the impact of GPU Dynamic Voltage and Frequency Scaling (DVFS) on the energy consumption and performance of deep learning. Our experiments cover a wide range of GPU architectures, DVFS settings, and DNN configurations. We observe that, compared to the default core frequency settings of three tested GPUs, the optimal core frequency can help conserve 8.7%$\sim$23.1% energy consumption for different DNN training cases. Regarding the inference, the benefits vary from 19.6%$\sim$26.4%. Our findings suggest that GPU DVFS has great potentials to help develop energy efficient DNN training/inference schemes.

Motivation & Objective

• To investigate the impact of GPU DVFS on energy consumption and performance in deep learning workloads.
• To identify optimal core frequency settings that balance performance and energy efficiency across diverse GPU architectures.
• To evaluate how different DNN configurations and convolution algorithms (GEMM, FFT, Winograd) respond to DVFS scaling.
• To explore whether memory frequency scaling significantly affects energy efficiency in modern GPUs.
• To provide empirical evidence for designing energy-aware DNN training and inference systems using DVFS.

Proposed method

• Conducted extensive experiments across three GPU architectures: Pascal P100, Volta V100, and Turing GTX 2080Ti.
• Measured performance, power, and energy consumption under varying core and memory frequency settings.
• Evaluated multiple DNN models (e.g., AlexNet, ResNet) and convolution algorithms (GEMM, FFT, Winograd) on different frameworks.
• Used cuDNN and TensorRT-optimized kernels to ensure realistic performance measurements on production-grade DNN workloads.
• Analyzed energy consumption curves to identify 'sweet spot' frequencies that minimize energy use while maintaining high performance.
• Compared results against default GPU frequency settings to quantify energy savings and performance gains.

Experimental results

Research questions

• RQ1 How does scaling GPU core frequency affect the performance and energy consumption of DNN training and inference?
• RQ2 Does the default GPU frequency setting provide optimal energy efficiency for DNN workloads?
• RQ3 How do different convolution algorithms (GEMM, FFT, Winograd) respond to DVFS in terms of energy savings?
• RQ4 What is the impact of memory frequency scaling on energy efficiency in modern GPUs?
• RQ5 Can layer-wise or workload-aware DVFS strategies further improve energy efficiency in DNN training?

Key findings

• Optimal core frequency settings reduced energy consumption by 8.7% to 23.1% during DNN training across three GPU models.
• For DNN inference, energy savings ranged from 19.6% to 26.4% compared to default settings.
• Performance improved by up to 38.2% for training and 33.0% for inference when core frequency was increased by 50% on the GTX 2080Ti.
• Energy consumption curves for P100 and V100 showed a valley trend, indicating a clear 'sweet spot' frequency for minimal energy use.
• GEMM-based convolution achieved 14.5% average energy savings at optimal frequency, FFT 12.6%, and Winograd 15.8%.
• Memory frequency scaling had negligible impact on energy efficiency, suggesting limited benefit for current GPU implementations.