[Paper Review] Efficient Federated Meta-Learning over Multi-Access Wireless Networks

Sheng Yue, Ju Ren+4 more|arXiv (Cornell University)|Aug 14, 2021

Wireless Networks and Protocols7 citations

TL;DR

This paper proposes NUFM, a non-uniform device selection algorithm for federated meta-learning that accelerates convergence by maximizing per-round global loss reduction. It further introduces URAL, a joint device selection and resource allocation strategy for multi-access wireless networks that optimizes convergence speed, wall-clock time, and energy efficiency, achieving up to 30% lower energy consumption and 20% faster convergence than baselines.

ABSTRACT

Federated meta-learning (FML) has emerged as a promising paradigm to cope with the data limitation and heterogeneity challenges in today's edge learning arena. However, its performance is often limited by slow convergence and corresponding low communication efficiency. In addition, since the available radio spectrum and IoT devices' energy capacity are usually insufficient, it is crucial to control the resource allocation and energy consumption when deploying FML in practical wireless networks. To overcome the challenges, in this paper, we rigorously analyze each device's contribution to the global loss reduction in each round and develop an FML algorithm (called NUFM) with a non-uniform device selection scheme to accelerate the convergence. After that, we formulate a resource allocation problem integrating NUFM in multi-access wireless systems to jointly improve the convergence rate and minimize the wall-clock time along with energy cost. By deconstructing the original problem step by step, we devise a joint device selection and resource allocation strategy to solve the problem with theoretical guarantees. Further, we show that the computational complexity of NUFM can be reduced from $O(d^2)$ to $O(d)$ (with the model dimension $d$) via combining two first-order approximation techniques. Extensive simulation results demonstrate the effectiveness and superiority of the proposed methods in comparison with existing baselines.

Motivation & Objective

• Address slow convergence and low communication efficiency in federated meta-learning (FML) due to uniform device selection.
• Tackle the trade-off between convergence speed, wall-clock time, and energy consumption in multi-access wireless edge networks.
• Design a joint device selection and resource allocation strategy that accounts for device heterogeneity, channel conditions, and limited radio and energy resources.
• Reduce computational complexity in IoT devices by applying first-order approximation techniques without sacrificing convergence performance.
• Provide theoretical guarantees for convergence and resource allocation optimization in practical wireless FML deployments.

Proposed method

• Develop NUFM by deriving a tight lower bound on per-round global loss reduction, enabling non-uniform device selection that maximizes this bound.
• Formulate a joint optimization problem integrating NUFM with CPU-cycle frequency control and resource block (RB) allocation to minimize wall-clock time and energy cost.
• Decompose the original non-convex mixed-integer non-linear programming (MINLP) problem into two sub-problems: computation time minimization and RB/power allocation optimization.
• Optimize CPU-cycle frequencies using analytical solutions that account for device heterogeneity and computation load.
• Design an iterative algorithm to solve the non-convex RB allocation and power control sub-problem, ensuring convergence with theoretical guarantees.
• Integrate two first-order approximation techniques to reduce NUFM’s computational complexity from $O(d^2)$ to $O(d)$, where $d$ is the model dimension.

Experimental results

Research questions

• RQ1 How can device selection in federated meta-learning be optimized to accelerate convergence beyond uniform random selection?
• RQ2 What is the impact of device heterogeneity, channel quality, and resource block allocation on the wall-clock time and energy consumption in wireless FML systems?
• RQ3 Can a joint optimization of device selection, CPU-cycle frequency, and transmission power achieve better trade-offs between convergence speed, latency, and energy efficiency?
• RQ4 How does the proposed method perform under varying wireless resource constraints, such as limited bandwidth or poor channel conditions?
• RQ5 To what extent can computational complexity in edge devices be reduced without degrading convergence performance in FML?

Key findings

• URAL achieves a convergence speed comparable to greedy selection methods (e.g., NUFM-Greedy), with wall-clock time and energy consumption reduced by up to 30% compared to baseline methods.
• The wall-clock time remains stable even as the number of resource blocks (RBs) increases from 1 to 50, indicating robustness to resource availability.
• Convergence speed does not monotonically decrease with more RBs; instead, it peaks at a moderate number of selected devices, confirming the existence of an optimal trade-off between information diversity and loss reduction per round.
• Improved channel quality (higher $h^{ ext{max}}$) reduces both transmission power and wall-clock time, with energy consumption decreasing significantly under good channel conditions.
• Tuning the weight parameters $ heta_1$ and $ heta_2$ allows explicit control over the trade-off between energy consumption and latency—higher $ heta_1$ reduces energy use at the cost of longer wall-clock time, and vice versa.
• The proposed first-order approximation techniques reduce the computational complexity of NUFM from $O(d^2)$ to $O(d)$, enabling practical deployment on energy-constrained IoT devices without sacrificing convergence performance.