[Paper Review] To Talk or to Work: Energy Efficient Federated Learning over Mobile Devices via the Weight Quantization and 5G Transmission Co-Design.

Rui Chen, Liang Li+4 more|arXiv (Cornell University)|Dec 21, 2020

Privacy-Preserving Technologies in Data32 references4 citations

TL;DR

This paper co-designs 5G wireless transmission and weight quantization to enable energy-efficient federated learning (FL) on heterogeneous mobile devices. By integrating multi-access edge computing (MEC) and formulating a mixed-integer programming problem, it jointly optimizes quantization levels and bandwidth allocation to minimize total energy consumption (computing + transmission) while preserving model accuracy and latency.

ABSTRACT

Federated learning (FL) is a new paradigm for large-scale learning tasks across mobile devices. However, practical FL deployment over resource constrained mobile devices confronts multiple challenges. For example, it is not clear how to establish an effective wireless network architecture to support FL over mobile devices. Besides, as modern machine learning models are more and more complex, the local on-device training/intermediate model update in FL is becoming too power hungry/radio resource intensive for mobile devices to afford. To address those challenges, in this paper, we try to bridge another recent surging technology, 5G, with FL, and develop a wireless transmission and weight quantization co-design for energy efficient FL over heterogeneous 5G mobile devices. Briefly, the 5G featured high data rate helps to relieve the severe communication concern, and the multi-access edge computing (MEC) in 5G provides a perfect network architecture to support FL. Under MEC architecture, we develop flexible weight quantization schemes to facilitate the on-device local training over heterogeneous 5G mobile devices. Observed the fact that the energy consumption of local computing is comparable to that of the model updates via 5G transmissions, we formulate the energy efficient FL problem into a mixed-integer programming problem to elaborately determine the quantization strategies and allocate the wireless bandwidth for heterogeneous 5G mobile devices. The goal is to minimize the overall FL energy consumption (computing + 5G transmissions) over 5G mobile devices while guaranteeing learning performance and training latency. Generalized Benders' Decomposition is applied to develop feasible solutions and extensive simulations are conducted to verify the effectiveness of the proposed scheme.

Motivation & Objective

• Address the challenge of high energy consumption in federated learning (FL) on resource-constrained mobile devices due to complex model training and communication overhead.
• Overcome the limitations of existing wireless network architectures in supporting scalable and efficient FL over mobile networks.
• Design a co-design framework that integrates 5G's high data rate and MEC capabilities with adaptive weight quantization to reduce energy consumption in FL.
• Minimize the total energy consumption (local computation + 5G transmission) across heterogeneous 5G mobile devices while ensuring learning performance and training latency constraints.
• Provide a scalable and practical solution for deploying energy-efficient FL in real-world 5G mobile environments.

Proposed method

• Leverages 5G's high data rate and multi-access edge computing (MEC) to provide a scalable network architecture for FL over mobile devices.
• Introduces flexible weight quantization schemes tailored to the computational and communication capabilities of heterogeneous 5G mobile devices.
• Models the energy efficiency problem as a mixed-integer programming (MIP) formulation that jointly optimizes quantization levels and wireless bandwidth allocation.
• Applies Generalized Benders' Decomposition to decompose and solve the complex MIP problem efficiently, enabling feasible and scalable solutions.
• Incorporates constraints on model accuracy and training latency to ensure practical deployment viability.
• Uses the MEC architecture to offload aggregation and reduce device-side computational load, improving energy efficiency.

Experimental results

Research questions

• RQ1 How can 5G wireless networks and FL be co-designed to reduce energy consumption in mobile federated learning?
• RQ2 What is the optimal trade-off between weight quantization precision and wireless bandwidth allocation to minimize total energy usage?
• RQ3 How can heterogeneous mobile devices with varying capabilities be efficiently supported in an energy-efficient FL framework?
• RQ4 What impact does joint optimization of quantization and transmission have on model convergence and training latency?
• RQ5 Can the proposed co-design approach maintain acceptable learning performance while significantly reducing energy consumption?

Key findings

• The proposed co-design framework achieves significant reductions in total energy consumption by jointly optimizing quantization and bandwidth allocation.
• Energy consumption from local computation is found to be comparable to that from 5G transmission, justifying joint optimization.
• The use of Generalized Benders' Decomposition enables efficient solution of the mixed-integer programming problem, ensuring scalability.
• Extensive simulations confirm that the proposed scheme maintains high model accuracy while minimizing energy usage across heterogeneous devices.
• The framework effectively balances training latency and energy efficiency, making it suitable for real-world 5G mobile deployments.
• The integration of MEC and adaptive quantization leads to a more scalable and energy-efficient FL architecture compared to conventional approaches.