[Paper Review] FedGNN: Federated Graph Neural Network for Privacy-Preserving Recommendation

Chuhan Wu, Fangzhao Wu+3 more|arXiv (Cornell University)|Feb 9, 2021

Privacy-Preserving Technologies in Data38 references131 citations

TL;DR

FedGNN trains GNN-based recommendations in a privacy-preserving federated setup, using local DP, pseudo interacted items, and privacy-preserving graph expansion to exploit high-order user-item interactions.

ABSTRACT

Graph neural network (GNN) is widely used for recommendation to model high-order interactions between users and items. Existing GNN-based recommendation methods rely on centralized storage of user-item graphs and centralized model learning. However, user data is privacy-sensitive, and the centralized storage of user-item graphs may arouse privacy concerns and risk. In this paper, we propose a federated framework for privacy-preserving GNN-based recommendation, which can collectively train GNN models from decentralized user data and meanwhile exploit high-order user-item interaction information with privacy well protected. In our method, we locally train GNN model in each user client based on the user-item graph inferred from the local user-item interaction data. Each client uploads the local gradients of GNN to a server for aggregation, which are further sent to user clients for updating local GNN models. Since local gradients may contain private information, we apply local differential privacy techniques to the local gradients to protect user privacy. In addition, in order to protect the items that users have interactions with, we propose to incorporate randomly sampled items as pseudo interacted items for anonymity. To incorporate high-order user-item interactions, we propose a user-item graph expansion method that can find neighboring users with co-interacted items and exchange their embeddings for expanding the local user-item graphs in a privacy-preserving way. Extensive experiments on six benchmark datasets validate that our approach can achieve competitive results with existing centralized GNN-based recommendation methods and meanwhile effectively protect user privacy.

Motivation & Objective

• Motivate privacy concerns with centralized user-item graphs in GNN-based recommendation.
• Propose FedGNN to enable federated, privacy-protecting training from decentralized user data.
• Enable high-order user-item interaction modeling without leaking private data.
• Demonstrate competitiveness with centralized GNN methods while preserving user privacy.

Proposed method

• Each user device learns embeddings and a local GNN from its own inferred user-item subgraph.
• Gradients (model and embedding) are uploaded to a server and aggregated via FedAvg for global updates.
• Two privacy mechanisms: pseudo interacted item sampling adds gradients from non-interacted items to mask real interactions; local differential privacy adds Laplacian noise to gradients.
• Privacy-preserving graph expansion uses a third-party server with homomorphic encryption to find anonymous neighbor users and enrich local graphs without exposing item IDs.
• Any GNN architecture (GCN, GGNN, GAT) can be used within FedGNN; the rating predictor uses dot-product to predict ratings.

Experimental results

Research questions

• RQ1 Can FedGNN learn effective GNN-based recommendations in a fully decentralized setting without sharing raw user-item data?
• RQ2 How can high-order user-item interactions be exploited privately to improve recommendations?
• RQ3 What are the privacy-utility trade-offs when using local DP and pseudo interacted item sampling in federated GNN training?
• RQ4 How does FedGNN compare to centralized GNN-based methods and other privacy-preserving baselines on standard datasets?

Key findings

• FedGNN achieves competitive RMSE with centralized GNN-based methods across six benchmark datasets.
• Incorporating high-order user-item interactions via the privacy-preserving expansion improves recommendation performance.
• FedGNN outperforms privacy-preserving baselines that do not leverage high-order information (e.g., FCF, FedMF).
• Localized DP and pseudo item sampling provide privacy protection with a controllable trade-off to accuracy, achieving 1-differential privacy under chosen settings.
• Fixed neighbor embeddings during training can yield slight accuracy gains over fully trainable neighbor embeddings.