[Paper Review] Robust Federated Contrastive Recommender System against Model Poisoning Attack

Wei Yuan, Chaoqun Yang+4 more|arXiv (Cornell University)|Mar 29, 2024

Spam and Phishing Detection4 citations

TL;DR

This paper proposes rCL4FedRec, a robust federated contrastive recommender system that enhances performance and resists model poisoning attacks by introducing a popularity-based contrastive regularizer. Unlike prior methods, it maintains client privacy by avoiding private parameter sharing and uses synthetic negative users and noise-augmented views to improve representation learning without compromising security or robustness.

ABSTRACT

Federated Recommender Systems (FedRecs) have garnered increasing attention recently, thanks to their privacy-preserving benefits. However, the decentralized and open characteristics of current FedRecs present two dilemmas. First, the performance of FedRecs is compromised due to highly sparse on-device data for each client. Second, the system's robustness is undermined by the vulnerability to model poisoning attacks launched by malicious users. In this paper, we introduce a novel contrastive learning framework designed to fully leverage the client's sparse data through embedding augmentation, referred to as CL4FedRec. Unlike previous contrastive learning approaches in FedRecs that necessitate clients to share their private parameters, our CL4FedRec aligns with the basic FedRec learning protocol, ensuring compatibility with most existing FedRec implementations. We then evaluate the robustness of FedRecs equipped with CL4FedRec by subjecting it to several state-of-the-art model poisoning attacks. Surprisingly, our observations reveal that contrastive learning tends to exacerbate the vulnerability of FedRecs to these attacks. This is attributed to the enhanced embedding uniformity, making the polluted target item embedding easily proximate to popular items. Based on this insight, we propose an enhanced and robust version of CL4FedRec (rCL4FedRec) by introducing a regularizer to maintain the distance among item embeddings with different popularity levels. Extensive experiments conducted on four commonly used recommendation datasets demonstrate that CL4FedRec significantly enhances both the model's performance and the robustness of FedRecs.

Motivation & Objective

• Address the challenge of data sparsity in federated recommender systems (FedRecs) due to highly limited on-device user data.
• Overcome the vulnerability of FedRecs to model poisoning attacks, where malicious clients manipulate global model updates.
• Design a contrastive learning framework compatible with standard FedRec protocols that does not require sharing private client parameters.
• Identify and resolve the counterintuitive issue that contrastive learning can worsen robustness by increasing embedding uniformity.
• Propose a popularity-aware regularizer to maintain distance between embeddings of items with different popularity levels, improving both performance and robustness.

Proposed method

• Construct synthetic negative users on the central server to serve as negative views for real clients’ user embeddings, enabling user-level contrastive learning without sharing private data.
• Apply lightweight uniform noise to user embeddings to generate positive view pairs, enhancing representation diversity.
• Use approximate optimization to generate two distinct views of item embeddings based on local interaction data, enabling item-level contrastive learning.
• Introduce a popularity-based contrastive regularizer that penalizes the model when embeddings of popular and unpopular items become too close, countering uniformity induced by contrastive learning.
• Ensure compatibility with standard FedRec training by keeping private user embeddings local and only sharing public item embeddings.
• Formulate the regularizer using popularity scores derived from global item interaction frequency, encouraging separation between high- and low-popularity item representations.

Experimental results

Research questions

• RQ1 Can contrastive learning be effectively applied in federated recommender systems without violating privacy constraints or requiring private parameter sharing?
• RQ2 Does the use of contrastive learning in FedRecs inadvertently increase vulnerability to model poisoning attacks, and if so, why?
• RQ3 To what extent can a popularity-based regularizer mitigate the negative impact of contrastive learning on model robustness?
• RQ4 How do different view construction strategies (e.g., synthetic users, noise augmentation, item view methods) affect the final recommendation performance?
• RQ5 What is the optimal number of synthetic users for balancing performance and robustness in the contrastive learning framework?

Key findings

• Contrastive learning in FedRecs increases embedding uniformity, making the model more vulnerable to model poisoning attacks, particularly because polluted target item embeddings become closer to popular items.
• The proposed rCL4FedRec framework significantly improves recommendation performance, achieving a Recall@20 of 0.04836 on MovieLens-1M, outperforming the baseline without the regularizer.
• Removing the popularity-based regularizer reduces Recall@20 to 0.04656, demonstrating that the regularizer is essential for both performance and robustness.
• The ablation study confirms that all components—user contrastive learning, item contrastive learning, and the regularizer—contribute meaningfully to performance, with the regularizer having the largest impact.
• The optimal number of synthetic users is 60 on MovieLens-1M and 10 on Amazon-Video, beyond which performance degrades due to over-regularization or noise in contrastive learning.
• rCL4FedRec is the only defense method that improves both model performance and robustness under poisoning attacks, outperforming state-of-the-art defenses like HiCS in both metrics.