[Paper Review] An Exploratory Analysis on Users' Contributions in Federated Learning

Jiyue Huang, Rania Talbi+4 more|arXiv (Cornell University)|Nov 13, 2020

Privacy-Preserving Technologies in Data43 references4 citations

TL;DR

This paper explores incentive mechanisms in federated learning to detect and mitigate malicious user contributions, particularly through label-flipping attacks. It evaluates three contribution measurement methods—Shapley value, Influence, and Reputation—showing a trade-off between computational efficiency and effectiveness in identifying low-quality updates, with Shapley value demonstrating the strongest alignment with theoretical expectations under adversarial conditions.

ABSTRACT

Federated Learning is an emerging distributed collaborative learning paradigm adopted by many of today's applications, e.g., keyboard prediction and object recognition. Its core principle is to learn from large amount of users data while preserving data privacy by design as collaborative users only need to share the machine learning models and keep data locally. The main challenge for such systems is to provide incentives to users to contribute high-quality models trained from their local data. In this paper, we aim to answer how well incentives recognize (in)accurate local models from honest and malicious users, and perceive their impacts on the model accuracy of federated learning systems. We first present a thorough survey on two contrasting perspectives: incentive mechanisms to measure the contribution of local models by honest users, and malicious users to deliberately degrade the overall model. We conduct simulation experiments to empirically demonstrate if existing contribution measurement schemes can disclose low-quality models from malicious users. Our results show there exists a clear tradeoff among measurement schemes in terms of the computational efficiency and effectiveness to distill the impact of malicious participants. We conclude this paper by discussing the research directions to design resilient contribution incentives.

Motivation & Objective

• To analyze how incentive mechanisms in federated learning recognize contributions from honest users versus malicious users who degrade model quality.
• To investigate the impact of adversarial behaviors—particularly label-flipping attacks—on model accuracy and contribution measurement reliability.
• To evaluate existing contribution measurement schemes (Shapley, Influence, Reputation) in detecting low-quality model updates from malicious participants.
• To identify research gaps in detecting free-riding and other non-model-quality-targeting attacks that exploit incentive mechanisms.
• To propose future research directions for designing attack-aware, resilient incentive mechanisms in federated learning systems.

Proposed method

• Conducts a survey of two contrasting perspectives: incentive mechanisms for honest contributors and adversarial behaviors that degrade model performance.
• Employs simulation experiments using a deep image classifier trained via federated learning to evaluate the impact of label-flipping attacks on model accuracy.
• Applies three contribution measurement techniques—Shapley value, Influence function, and Reputation-based scoring—to assess model update quality.
• Measures the marginal loss contribution of each user’s update to evaluate how well each metric detects malicious participants.
• Uses secure aggregation to combine model updates without exposing raw data, preserving privacy while enabling centralized evaluation.
• Compares metrics under varying attack intensities (flipping levels) to assess robustness and computational cost.

Experimental results

Research questions

• RQ1 How do existing contribution measurement schemes perform in detecting low-quality model updates from malicious users in federated learning?
• RQ2 What trade-offs exist between computational efficiency and detection effectiveness in incentive mechanisms under adversarial conditions?
• RQ3 How do different attack strategies—such as label flipping and free-riding—affect the reliability of contribution measurement in federated learning?
• RQ4 To what extent do Shapley value, Influence, and Reputation metrics align with theoretical expectations in identifying malicious contributions?
• RQ5 What new attack vectors targeting incentive mechanisms exist, and how can they be detected or mitigated?

Key findings

• A clear trade-off exists between computational efficiency and detection effectiveness among contribution measurement schemes, with Shapley value offering the most accurate detection despite higher cost.
• Shapley value outperforms Influence and Reputation in detecting malicious users under varying label-flipping levels, showing the strongest theoretical consistency.
• Label-flipping attacks significantly degrade model accuracy and can evade detection by less sophisticated metrics, especially Reputation and Influence.
• Free-riding attacks—where users submit fake updates without actual training—remain largely unexplored but pose a critical threat to incentive fairness.
• Existing metrics struggle to distinguish between low-quality honest updates and malicious ones, especially when attack intensity is moderate.
• Blockchain-based contribution logging may improve traceability and enable detection of malicious users through verifiable parameter records.