[Paper Review] Energy Efficiency of Training Neural Network Architectures: An Empirical Study

Yinlena Xu, Silverio Martínez‐Fernández+2 more|arXiv (Cornell University)|Feb 2, 2023

Green IT and Sustainability11 citations

TL;DR

The paper empirically analyzes how CNN architectures (VGG16/19, ResNet50) and training locations affect energy use and CO2 emissions during image-classification model training, comparing hardware and software energy measurements and trade-offs with accuracy.

ABSTRACT

The evaluation of Deep Learning models has traditionally focused on criteria such as accuracy, F1 score, and related measures. The increasing availability of high computational power environments allows the creation of deeper and more complex models. However, the computations needed to train such models entail a large carbon footprint. In this work, we study the relations between DL model architectures and their environmental impact in terms of energy consumed and CO$_2$ emissions produced during training by means of an empirical study using Deep Convolutional Neural Networks. Concretely, we study: (i) the impact of the architecture and the location where the computations are hosted on the energy consumption and emissions produced; (ii) the trade-off between accuracy and energy efficiency; and (iii) the difference on the method of measurement of the energy consumed using software-based and hardware-based tools.

Motivation & Objective

• Assess whether CNN architecture impacts energy consumption and emissions during training.
• Quantify the trade-off between model accuracy and training energy/CO2 footprint.
• Compare software-based energy profiling with hardware-based measurements (wattmeter).
• Analyze how dataset choice and hosting location influence energy metrics.
• Provide guidance for greener CNN training in computer vision.

Proposed method

• Use transfer learning with VGG16, VGG19, and ResNet50 on CIFAR-10 and MNIST datasets.
• Measure energy and CO2 emissions using CodeCarbon profiler (software-based) and a wattmeter (hardware-based).
• Compute FLOPs with keras-flops to approximate computational workload.
• Calculate Score = Accuracy / Energy to study accuracy-energy trade-offs.
• Analyze energy metrics across three locations and multiple architectures via violin/box plots and non-parametric statistics.
• Perform Kruskal-Wallis tests to assess architecture and location effects on emissions and energy.

Experimental results

Research questions

• RQ1 RQ1: Does CNN architecture affect energy consumption and emissions during training?
• RQ2 RQ2: What is the relationship between model accuracy and the energy required to train the model?
• RQ3 RQ3: How do software-based and hardware-based energy measurement methods compare for training energy efficiency?

Key findings

• CIFAR-10 training generally yields lower emissions and energy than MNIST across architectures.
• VGG16 and VGG19 tend to produce similar emissions and energy, both lower than ResNet50.
• Location of computation influences emissions, with Taiwan showing higher emissions than US locations; energy consumed showed no location difference.
• Energy consumption and FLOPs correlate with dataset choice; CIFAR-10 requires fewer FLOPs and less energy than MNIST.
• Score (Accuracy/Energy) varies by location and architecture, illustrating trade-offs between accuracy gains and energy cost.
• Software-based profiling (CodeCarbon) correlates strongly with hardware wattmeter measurements (rho ≈ 0.94).