[Paper Review] Energy Efficient Federated Learning Over Wireless Communication Networks

Zhaohui Yang, Mingzhe Chen+3 more|arXiv (Cornell University)|Nov 6, 2019

Privacy-Preserving Technologies in Data42 references53 citations

TL;DR

This paper proposes an energy minimization framework for federated learning over wireless networks, jointly optimizing local computation and transmission resources to meet latency constraints, with an iterative algorithm achieving up to 59.5% energy reduction.

ABSTRACT

In this paper, the problem of energy efficient transmission and computation resource allocation for federated learning (FL) over wireless communication networks is investigated. In the considered model, each user exploits limited local computational resources to train a local FL model with its collected data and, then, sends the trained FL model to a base station (BS) which aggregates the local FL model and broadcasts it back to all of the users. Since FL involves an exchange of a learning model between users and the BS, both computation and communication latencies are determined by the learning accuracy level. Meanwhile, due to the limited energy budget of the wireless users, both local computation energy and transmission energy must be considered during the FL process. This joint learning and communication problem is formulated as an optimization problem whose goal is to minimize the total energy consumption of the system under a latency constraint. To solve this problem, an iterative algorithm is proposed where, at every step, closed-form solutions for time allocation, bandwidth allocation, power control, computation frequency, and learning accuracy are derived. Since the iterative algorithm requires an initial feasible solution, we construct the completion time minimization problem and a bisection-based algorithm is proposed to obtain the optimal solution, which is a feasible solution to the original energy minimization problem. Numerical results show that the proposed algorithms can reduce up to 59.5% energy consumption compared to the conventional FL method.

Motivation & Objective

• Motivate energy-efficient FL over wireless networks under privacy and resource constraints.
• Model the joint impact of local computation and wireless transmission on FL latency and energy.
• Develop an iterative resource allocation algorithm to minimize total energy under a latency constraint.

Proposed method

• Model FL over FDMA cellular networks with local computation and uplink transmission of local models.
• Derive FL convergence rate under local computation accuracy and global aggregation.
• Formulate a joint energy minimization problem over time, bandwidth, power, computation frequency, and learning accuracy.
• Propose a low-complexity iterative algorithm with closed-form updates for each variable.
• Provide a feasibility-finding step via a completion-time minimization problem and a bisection-based solution.
• Present a detailed resource allocation workflow including Dinkelbach-based optimization for the learning accuracy parameter.

Experimental results

Research questions

• RQ1 How does FL performance and convergence depend on local computation accuracy and learning dynamics under wireless resource constraints?
• RQ2 What is the minimum total energy to complete FL given latency constraints and resource limits?
• RQ3 How can time, bandwidth, power, and computation frequency be jointly optimized to minimize energy while satisfying latency and data transmission requirements?
• RQ4 Can a feasible initial solution be efficiently found to enable the energy minimization algorithm?
• RQ5 What energy savings can be achieved compared to conventional FL without joint optimization?

Key findings

• An energy-efficient joint computation and transmission resource allocation scheme reduces total energy consumption compared to conventional FL methods by up to 59.5%.
• The paper derives the FL convergence rate when incorporating local computation accuracy and global aggregation in a wireless setting.
• An iterative algorithm with closed-form solutions for time, bandwidth, power, computation frequency, and learning accuracy is proposed for the energy minimization problem.
• A completion-time minimization approach with a bisection-based algorithm provides feasible starting points for the energy optimization.
• The optimization framework accounts for both local computation energy and transmission energy, under a latency constraint, achieving substantial energy savings.