[Paper Review] Hierarchically Fair Federated Learning

Jingfeng Zhang, Cheng Li+2 more|arXiv (Cornell University)|Apr 22, 2020

Privacy-Preserving Technologies in Data30 references43 citations

TL;DR

Proposes Hierarchically Fair Federated Learning (HFFL) to reward agents proportionally to their contribution levels in federated setups, with an extended version HFFL+ that supports heterogeneous models. It demonstrates fairness and improved generalization with empirical evaluation on multiple datasets.

ABSTRACT

When the federated learning is adopted among competitive agents with siloed datasets, agents are self-interested and participate only if they are fairly rewarded. To encourage the application of federated learning, this paper employs a management strategy, i.e., more contributions should lead to more rewards. We propose a novel hierarchically fair federated learning (HFFL) framework. Under this framework, agents are rewarded in proportion to their pre-negotiated contribution levels. HFFL+ extends this to incorporate heterogeneous models. Theoretical analysis and empirical evaluation on several datasets confirm the efficacy of our frameworks in upholding fairness and thus facilitating federated learning in the competitive settings.

Motivation & Objective

• Motivate fairness in federated learning when agents are self-interested and possess siloed datasets.
• Propose a contribution-based reward mechanism where more contributions yield greater rewards.
• Introduce HFFL to learn models at multiple contribution levels and demonstrate proportional fairness.
• Extend to HFFL+ to allow heterogeneous model architectures across levels.
• Provide theoretical and empirical justification that more data leads to better generalization and higher rewards.

Proposed method

• Classify agents into multiple contribution levels based on publicly verifiable factors (data volume, data quality, collection cost, etc.).
• At each level, train a shared model f_l using data from all levels up to that point, with higher levels contributing more data to higher-fidelity models.
• Use FedAvg-style parameter aggregation across agents within each level to update local and global models.
• Provide an extension HFFL+ that runs HFFL with different model architectures and selects the best-performing model per level.
• Offer a theoretical bound linking more data to lower generalization error (via standard generalization arguments).
• Validate fairness and performance through experiments on four datasets with fixed 3-level hierarchies.

Experimental results

Research questions

• RQ1 Can agents be induced to participate in federated learning when rewards are proportional to contribution levels?
• RQ2 Does hierarchical contribution-based learning (HFFL) achieve proportional fairness and improved models for higher-contributing agents?
• RQ3 Does extending to heterogeneous models (HFFL+) further improve performance across levels while preserving fairness?
• RQ4 Is higher data availability at upper levels associated with better generalization/error bounds in the federated setting?
• RQ5 Are publicly verifiable contribution factors sufficient to prevent model-dependent or Shapley-based valuation pitfalls in incentive design?

Key findings

• Higher-contribution-level models achieve better performance than lower-level models in all four datasets tested.
• HFFL+ consistently yields higher model scores than HFFL, by selecting architecture per level.
• More data at higher levels leads to models with better generalization potential according to the theoretical analysis.
• Empirical results on ADULT, MNIST, Fashion MNIST, and IMDB datasets confirm the proposed fairness mechanism and its practicality across tasks.