[Paper Review] Fair Resource Allocation in Federated Learning

Li Tian, Maziar Sanjabi+2 more|arXiv (Cornell University)|May 25, 2019

Privacy-Preserving Technologies in Data45 references352 citations

TL;DR

The paper introduces q-Fair Federated Learning (q-FFL) and a FedAvg-style solver (q-FedAvg) to achieve a more uniform device-level accuracy distribution in federated learning, balancing fairness and overall performance, with demonstrated efficiency gains.

ABSTRACT

Federated learning involves training statistical models in massive, heterogeneous networks. Naively minimizing an aggregate loss function in such a network may disproportionately advantage or disadvantage some of the devices. In this work, we propose q-Fair Federated Learning (q-FFL), a novel optimization objective inspired by fair resource allocation in wireless networks that encourages a more fair (specifically, a more uniform) accuracy distribution across devices in federated networks. To solve q-FFL, we devise a communication-efficient method, q-FedAvg, that is suited to federated networks. We validate both the effectiveness of q-FFL and the efficiency of q-FedAvg on a suite of federated datasets with both convex and non-convex models, and show that q-FFL (along with q-FedAvg) outperforms existing baselines in terms of the resulting fairness, flexibility, and efficiency.

Motivation & Objective

• Motivate fairness across devices in federated learning due to data heterogeneity and system variability.
• Define a flexible fairness-aware objective that trades off average accuracy against uniformity of performance across devices.
• Develop scalable optimization methods tailored to federated settings that minimize communication and handle varying device participation.
• Establish theoretical connections to alpha-fairness and compare with existing fairness approaches in federated learning.
• Empirically validate that q-FFL achieves more uniform device performance while maintaining overall accuracy.

Proposed method

• Propose q-Fair Federated Learning (q-FFL) with objective f_q(w)=sum_k (p_k/(q+1)) F_k(w)^{q+1}, parameterized by q to control fairness.
• Relate q-FFL to alpha-fairness to enable a tunable fairness-accuracy trade-off.
• Introduce q-FedSGD, a dynamic step-size Federated SGD method leveraging upper bounds on local Lipschitz constants to update models.
• Extend to q-FedAvg, a communication-efficient FedAvg-like method that uses dynamic weighting of local updates based on Lipschitz bounds.
• Provide a practical scheme to estimate Lipschitz constants from q=0 and reuse them for q>0 to avoid per-q hyperparameter tuning.
• Demonstrate that q-FedAvg achieves faster convergence in communication rounds than q-FedSGD and standard FedSGD while solving q-FFL.

Experimental results

Research questions

• RQ1 How can we define fairness in the distribution of model performance across devices in federated learning?
• RQ2 Can a tunable objective (q-FFL) balance average accuracy with uniformity of device-level performance across heterogeneous data distributions?
• RQ3 Do scalable optimization methods (q-FedAvg, q-FedSGD) solve the q-FFL objective efficiently in federated settings?
• RQ4 How does q-FFL compare with existing fairness approaches (AFL, uniform device weighting) in terms of accuracy and fairness metrics?
• RQ5 Can q-FFL be extended to other domains like meta-learning to achieve fair initializations across tasks?

Key findings

• q-FFL reduces the variance of accuracies across devices by 45% on average while maintaining the same overall average accuracy.
• q-FedAvg solves the q-FFL objective orders-of-magnitude faster than baselines, demonstrating improved communication efficiency.
• Compared to AFL and uniform sampling, q-FFL achieves more fair testing accuracy distributions without sacrificing average performance.
• On multiple federated datasets with convex and non-convex models, increasing q yields more uniform performance across devices (e.g., lower variance) with comparable averages.
• q-FFL is flexible: larger q yields tighter fairness control and can outperform minimax fairness baselines in worst-device metrics on certain datasets.
• The approach extends to meta-learning, producing fair initializations that reduce variance across tasks.