[Paper Review] Resource Allocation of Federated Learning for the Metaverse with Mobile Augmented Reality

Xinyu Zhou, Chang Liu+1 more|arXiv (Cornell University)|Nov 16, 2022

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes a joint resource allocation framework for federated learning (FL) in mobile augmented reality (MAR) systems within the Metaverse, optimizing bandwidth, transmission power, CPU frequency, and video resolution to minimize a weighted combination of energy, latency, and model accuracy. The algorithm outperforms benchmarks in all metrics, especially under strict time constraints.

ABSTRACT

The Metaverse has received much attention recently. Metaverse applications via mobile augmented reality (MAR) require rapid and accurate object detection to mix digital data with the real world. Federated learning (FL) is an intriguing distributed machine learning approach due to its privacy-preserving characteristics. Due to privacy concerns and the limited computation resources on mobile devices, we incorporate FL into MAR systems of the Metaverse to train a model cooperatively. Besides, to balance the trade-off between energy, execution latency and model accuracy, thereby accommodating different demands and application scenarios, we formulate an optimization problem to minimize a weighted combination of total energy consumption, completion time and model accuracy. Through decomposing the non-convex optimization problem into two subproblems, we devise a resource allocation algorithm to determine the bandwidth allocation, transmission power, CPU frequency and video frame resolution for each participating device. We further present the convergence analysis and computational complexity of the proposed algorithm. Numerical results show that our proposed algorithm has better performance (in terms of energy consumption, completion time and model accuracy) under different weight parameters compared to existing benchmarks.

Motivation & Objective

• Address the challenge of training accurate, privacy-preserving object detection models in Metaverse-enabled mobile augmented reality (MAR) applications.
• Overcome limitations in data availability and privacy concerns by integrating federated learning (FL) into MAR systems.
• Formulate a joint optimization problem balancing energy consumption, completion time, and model accuracy in FL-MAR systems.
• Design a resource allocation algorithm that dynamically assigns bandwidth, transmission power, CPU frequency, and video resolution per device.
• Ensure adaptability to diverse application scenarios by tuning weight parameters in the multi-objective optimization.

Proposed method

• Formulate a non-convex multi-objective optimization problem minimizing a weighted sum of total energy, completion time, and model accuracy.
• Decompose the problem into two subproblems: transmission resource allocation and computation resource allocation, solved iteratively.
• Use the Dinkelbach method to handle the fractional programming nature of the energy-latency trade-off.
• Fix CPU frequency per device based on a derived expression from latency and computation constraints to ensure feasibility.
• Apply sequential convex approximation (SCA) to handle non-convexity in the transmission power and bandwidth optimization.
• Integrate video frame resolution as an optimization variable to reduce data transmission while maintaining model performance.

Figure 1 : Federated learning with mobile augmented reality (FL-MAR) for the Metaverse.

Experimental results

Research questions

• RQ1 How can resource allocation in FL-MAR systems be jointly optimized across bandwidth, transmission power, CPU frequency, and video resolution to balance energy, latency, and accuracy?
• RQ2 What is the impact of varying weight parameters on the trade-off between energy efficiency, training completion time, and model accuracy in FL-MAR systems?
• RQ3 How does the proposed algorithm compare to baseline schemes that optimize only transmission or only computation energy?
• RQ4 In what scenarios does the proposed algorithm significantly outperform existing methods, especially under strict time constraints?
• RQ5 What is the convergence behavior and computational complexity of the proposed resource allocation algorithm?

Key findings

• The proposed algorithm achieves lower total energy consumption than both the Computation-only and Transmission-only schemes across all tested weight parameters.
• Transmission energy dominates total energy consumption, making transmission resource optimization critical for energy savings.
• The performance gap between the proposed algorithm and the benchmark (Scheme 1) increases as the maximum completion time decreases, indicating superior performance under tight latency constraints.
• At a completion time of 150 seconds, the proposed algorithm shows a non-monotonic energy trend due to the initial high transmission power setting, but still outperforms the baseline.
• The algorithm converges reliably and demonstrates manageable computational complexity, making it suitable for real-time deployment in MAR-enabled Metaverse systems.
• Joint optimization of video frame resolution significantly improves the overall system performance, as removing this variable reduces effectiveness.

Figure 2 : Illustration of the 1st convolutional layer in a CNN. In this example, padding is 0 and stride is 1. $c_{0}$ is the default number of channels (e.g., in a color image, we have three channels). $c_{1}$ is the number of kernels, the same as the number of output channels. $m_{1}$ is calculat