[Paper Review] Maximizing Global Model Appeal in Federated Learning

Yae Jee Cho, Divyansh Jhunjhunwala+3 more|arXiv (Cornell University)|May 30, 2022

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper introduces MaxFL, a novel federated learning framework that maximizes global model appeal—the fraction of clients whose individual performance requirements are met—by using a client requirement-aware weighted aggregation of model updates. MaxFL achieves 22–40% higher test accuracy on seen clients and 18–50% on unseen clients compared to state-of-the-art FL methods, while providing convergence guarantees under non-convex, partial participation settings.

ABSTRACT

Federated learning typically considers collaboratively training a global model using local data at edge clients. Clients may have their own individual requirements, such as having a minimal training loss threshold, which they expect to be met by the global model. However, due to client heterogeneity, the global model may not meet each client's requirements, and only a small subset may find the global model appealing. In this work, we explore the problem of the global model lacking appeal to the clients due to not being able to satisfy local requirements. We propose MaxFL, which aims to maximize the number of clients that find the global model appealing. We show that having a high global model appeal is important to maintain an adequate pool of clients for training, and can directly improve the test accuracy on both seen and unseen clients. We provide convergence guarantees for MaxFL and show that MaxFL achieves a $22$-$40\%$ and $18$-$50\%$ test accuracy improvement for the training clients and unseen clients respectively, compared to a wide range of FL modeling approaches, including those that tackle data heterogeneity, aim to incentivize clients, and learn personalized or fair models.

Motivation & Objective

• To address the critical issue of low client participation in federated learning due to global models failing to meet individual client performance requirements.
• To define and formalize the concept of 'global model appeal' as the fraction of clients whose local requirements are satisfied by the global model.
• To develop a training framework that explicitly maximizes this appeal, enhancing both client retention and model generalization.
• To provide theoretical convergence guarantees for the proposed method under realistic FL conditions, including partial client participation and non-convex objectives.
• To empirically validate MaxFL’s superiority over existing methods in fairness, personalization, and client incentive mechanisms.

Proposed method

• MaxFL formulates a global optimization objective that maximizes the number of clients whose local loss is below a client-specific threshold, defined as the global model appeal (GM-Appeal).
• It uses a differentiable, sigmoid-based relaxation to approximate the indicator function of client satisfaction, enabling gradient-based optimization of the client-weighted aggregation scheme.
• The method employs a client-weighted aggregation rule where client update contributions are scaled based on their expected contribution to increasing GM-Appeal, derived from empirical local losses.
• It introduces a theoretical learning rate adaptation strategy that increases convergence speed in flat regions of the objective function, improving optimization efficiency.
• The framework is stateless and supports partial client participation, making it practical for real-world federated settings with dynamic client availability.
• It replaces true local losses with empirical losses in the optimization objective, justified empirically by showing minimal deviation between true and empirical losses in trained models.

Experimental results

Research questions

• RQ1 How can federated learning be restructured to maximize the number of clients that find the global model appealing?
• RQ2 What impact does high global model appeal have on test accuracy for both seen and unseen clients?
• RQ3 Can a client requirement-aware aggregation strategy outperform existing methods focused on fairness, personalization, or variance reduction?
• RQ4 What theoretical convergence guarantees can be provided for a non-convex, client-weighted, appeal-maximizing federated learning algorithm?
• RQ5 How robust is the proposed method under dynamic client participation, including opt-outs and Byzantine clients?

Key findings

• MaxFL increases global model appeal (GM-Appeal) to at least 0.40 across all evaluated datasets, significantly outperforming fairness-focused methods like q-FFL, which achieve less than 0.01 GM-Appeal.
• MaxFL achieves a 22–40% improvement in test accuracy on seen clients compared to a wide range of FL baselines, including personalized, fair, and variance-reduction methods.
• For unseen clients, MaxFL delivers an 18–50% accuracy improvement, demonstrating superior generalization due to a larger, more diverse client pool enabled by high appeal.
• The method maintains strong performance even when clients opt out or when Byzantine clients participate, showing robustness in dynamic environments.
• Empirical validation confirms that true local losses are nearly identical to empirical losses in trained models, supporting the key relaxation used in MaxFL’s optimization.
• The theoretical learning rate adaptation in MaxFL increases convergence speed in flat regions, as shown in the mean estimation example, validating the adaptive learning rate strategy.