[Paper Review] Vertical Federated Graph Neural Network for Recommender System

Peihua Mai, Yan Pang|arXiv (Cornell University)|Mar 10, 2023

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes VerFedGNN, the first vertical federated graph neural network for recommender systems, enabling privacy-preserving collaborative training across organizations with disjoint user-item data. By using random projection for neighbor embedding aggregation and ternary quantization for gradient transmission, VerFedGNN achieves competitive recommendation accuracy while significantly enhancing privacy protection against de-anonymization attacks.

ABSTRACT

Conventional recommender systems are required to train the recommendation model using a centralized database. However, due to data privacy concerns, this is often impractical when multi-parties are involved in recommender system training. Federated learning appears as an excellent solution to the data isolation and privacy problem. Recently, Graph neural network (GNN) is becoming a promising approach for federated recommender systems. However, a key challenge is to conduct embedding propagation while preserving the privacy of the graph structure. Few studies have been conducted on the federated GNN-based recommender system. Our study proposes the first vertical federated GNN-based recommender system, called VerFedGNN. We design a framework to transmit: (i) the summation of neighbor embeddings using random projection, and (ii) gradients of public parameter perturbed by ternary quantization mechanism. Empirical studies show that VerFedGNN has competitive prediction accuracy with existing privacy preserving GNN frameworks while enhanced privacy protection for users' interaction information.

Motivation & Objective

• Address the challenge of training collaborative GNN-based recommender systems when user-item interaction data are isolated across organizations due to privacy and commercial constraints.
• Design a vertical federated learning framework that enables secure model training without sharing raw user interaction data or graph topology.
• Protect user interaction privacy during embedding propagation across subgraphs while maintaining model performance.
• Develop and evaluate a de-anonymization attack to demonstrate the vulnerability of existing federated GNN frameworks and validate the effectiveness of the proposed privacy mechanisms.

Proposed method

• Employ random projection to compress and transmit the summation of neighbor embeddings across clients, reducing communication cost and preserving privacy.
• Use ternary quantization to perturb and transmit gradients of public model parameters, minimizing information leakage while maintaining model update fidelity.
• Design a vertical federated learning framework where each client holds a different set of items but shares the same user base, enabling collaborative learning without data sharing.
• Integrate a server that aggregates projected neighbor embeddings and coordinates model updates across clients using secure, privacy-preserving communication protocols.
• Apply theoretical privacy analysis to demonstrate that the proposed method protects user interaction data under differential privacy and k-anonymity principles.
• Implement a de-anonymization attack to evaluate the robustness of the framework by inferring cross-graph interactions from gradients, validating the effectiveness of ternary quantization.

Experimental results

Research questions

• RQ1 Can a vertical federated GNN framework be designed to enable collaborative training of recommender systems without sharing raw user-item interaction data?
• RQ2 How effective is random projection in preserving privacy while enabling accurate embedding propagation across disjoint subgraphs?
• RQ3 To what extent does ternary quantization of gradients reduce the risk of de-anonymization attacks compared to unquantized or noise-added gradients?
• RQ4 How does VerFedGNN compare to centralized GNNs in terms of recommendation accuracy under privacy constraints?
• RQ5 What is the impact of the proposed framework on communication cost and scalability in large-scale datasets?

Key findings

• VerFedGNN achieves competitive recommendation accuracy, with RMSE of 1.312 on the Yelp dataset—only a 0.01 drop from the centralized GNN (1.302) and a significant improvement over matrix factorization (1.41).
• The use of random projection reduces attack accuracy in de-anonymization attacks to 0.0037 F1 on BookCrossing (vs. 0.0305 without projection), demonstrating strong privacy protection.
• Ternary quantization reduces gradient-based de-anonymization attack F1 to 0.0057 at 20% attack rate, significantly lower than FedPerGNN (0.0168) and FedSage+ (0.0220).
• Communication cost is manageable at 660.9MB per party per round (6609MB total), and transmission time can be reduced to 4.5 seconds with 5% user sampling per round.
• The framework is scalable and successfully deployed on a large-scale Yelp dataset with over 6.9 million ratings, 1.9 million users, and 150k items.
• Empirical results confirm that VerFedGNN maintains high performance while offering stronger privacy guarantees than existing federated GNN frameworks, especially under gradient inference attacks.