[Paper Review] Incentive Mechanism for Privacy-Preserving Federated Learning.

Shuyuan Zheng, Yang Cao+1 more|arXiv (Cornell University)|Jun 8, 2021

Privacy-Preserving Technologies in Data33 references4 citations

TL;DR

This paper proposes FL-Market, a novel framework that jointly addresses incentive mechanisms and privacy protection in federated learning by quantifying data owners' privacy loss via local differential privacy (LDP) and offering personalized payments. It uses a deep learning-empowered auction mechanism to incentivize participation with less noisy gradients, achieving both higher model utility and tailored privacy protection.

ABSTRACT

Federated learning (FL) is an emerging paradigm for machine learning, in which data owners can collaboratively train a model by sharing gradients instead of their raw data. Two fundamental research problems in FL are incentive mechanism and privacy protection. The former focuses on how to incentivize data owners to participate in FL. The latter studies how to protect data owners' privacy while maintaining high utility of trained models. However, incentive mechanism and privacy protection in FL have been studied separately and no work solves both problems at the same time. In this work, we address the two problems simultaneously by an FL-Market that incentivizes data owners' participation by providing appropriate payments and privacy protection. FL-Market enables data owners to obtain compensation according to their privacy loss quantified by local differential privacy (LDP). Our insight is that, by meeting data owners' personalized privacy preferences and providing appropriate payments, we can (1) incentivize privacy risk-tolerant data owners to set larger privacy parameters (i.e., gradients with less noise) and (2) provide preferred privacy protection for privacy risk-averse data owners. To achieve this, we design a personalized LDP-based FL framework with a deep learning-empowered auction mechanism for incentivizing trading gradients with less noise and optimal aggregation mechanisms for model updates. Our experiments verify the effectiveness of the proposed framework and mechanisms.

Motivation & Objective

• To address the gap in existing federated learning systems that treat incentive mechanisms and privacy protection as separate problems.
• To design a unified framework that simultaneously incentivizes data owners to participate and protects their privacy according to individual risk preferences.
• To enable data owners to receive compensation proportional to their privacy loss, quantified using local differential privacy (LDP).
• To develop an auction mechanism that encourages participation by rewarding users who contribute gradients with less noise (i.e., higher utility).
• To ensure optimal model aggregation by integrating personalized privacy levels and payment strategies into the training process.

Proposed method

• Introduces FL-Market, a market-based framework that enables data owners to trade gradients in exchange for payments based on their privacy loss.
• Employs local differential privacy (LDP) to quantify and manage individual data owners' privacy risk, allowing personalized privacy parameters.
• Designs a deep learning-empowered auction mechanism to determine optimal payments based on the quality (noise level) of gradients submitted by participants.
• Uses an optimal aggregation mechanism that weights model updates according to both privacy levels and contribution quality to improve global model utility.
• Integrates personalized privacy preferences into the training loop, allowing privacy-averse users to set stricter privacy parameters while still being compensated.

Experimental results

Research questions

• RQ1 How can we design an incentive mechanism in federated learning that accounts for individual data owners' privacy preferences?
• RQ2 What is the impact of personalized privacy loss quantification via LDP on model utility and participant participation?
• RQ3 Can an auction-based payment system effectively incentivize data owners to contribute higher-quality (less noisy) gradients?
• RQ4 How does the integration of personalized privacy and payment mechanisms affect the overall performance and convergence of federated models?
• RQ5 What trade-offs exist between privacy protection, model utility, and participant incentives in a federated learning setting?

Key findings

• The proposed FL-Market framework successfully incentivizes data owners to participate in federated learning by aligning payments with their privacy loss, as measured by local differential privacy (LDP).
• Privacy risk-tolerant participants are more likely to contribute gradients with less noise due to higher compensation, improving model utility.
• Privacy risk-averse participants are protected by stricter LDP parameters, ensuring their data remains well-protected without compromising participation.
• The deep learning-empowered auction mechanism effectively identifies and rewards high-quality gradient contributions, leading to faster convergence and better model accuracy.
• The optimal aggregation mechanism enhances model performance by weighting updates according to both privacy levels and data quality, resulting in improved global model utility.