[Paper Review] FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks

Chaoyang He, Keshav Balasubramanian+12 more|arXiv (Cornell University)|Apr 14, 2021

Advanced Graph Neural NetworksComputer Science102 references103 citations

TL;DR

FedGraphNN introduces an open federated learning benchmark system for graph neural networks, spanning diverse datasets, GNN models, and FL algorithms, and provides an efficient, secure, and modular system for cross-silo federated graph learning.

ABSTRACT

Graph Neural Network (GNN) research is rapidly growing thanks to the capacity of GNNs in learning distributed representations from graph-structured data. However, centralizing a massive amount of real-world graph data for GNN training is prohibitive due to privacy concerns, regulation restrictions, and commercial competitions. Federated learning (FL), a trending distributed learning paradigm, provides possibilities to solve this challenge while preserving data privacy. Despite recent advances in vision and language domains, there is no suitable platform for the FL of GNNs. To this end, we introduce FedGraphNN, an open FL benchmark system that can facilitate research on federated GNNs. FedGraphNN is built on a unified formulation of graph FL and contains a wide range of datasets from different domains, popular GNN models, and FL algorithms, with secure and efficient system support. Particularly for the datasets, we collect, preprocess, and partition 36 datasets from 7 domains, including both publicly available ones and specifically obtained ones such as hERG and Tencent. Our empirical analysis showcases the utility of our benchmark system, while exposing significant challenges in graph FL: federated GNNs perform worse in most datasets with a non-IID split than centralized GNNs; the GNN model that attains the best result in the centralized setting may not maintain its advantage in the FL setting. These results imply that more research efforts are needed to unravel the mystery behind federated GNNs. Moreover, our system performance analysis demonstrates that the FedGraphNN system is computationally efficient and secure to large-scale graphs datasets. We maintain the source code at https://github.com/FedML-AI/FedGraphNN.

Motivation & Objective

• Provide a unified federated graph learning framework (graph FL) and diverse task settings (graph-level, subgraph-level, node-level).
• Assemble and preprocess 36 graph datasets from 7 domains to simulate realistic non-IID federated scenarios.
• Deliver an efficient, secure, and modular FedGraphNN benchmark system enabling reproducible experiments.
• Evaluate how federated GNNs perform vs centralized baselines and uncover key challenges in graph FL.

Proposed method

• Formulates FedGraphNN as a distributed optimization problem over K clients with local objectives weighted by data proportions (F(W)=sum_k (N^(k)/N) f^(k)(W)).
• Adopts an inductive GNN framework (MPNN) with two phases: message-passing and readout, allowing various GNNs (GCN, GAT, GraphSAGE, SGC, GIN).
• Supports FL algorithms (FedAvg, FedOPT, etc.) and secure aggregation (LightSecAgg) for privacy-preserving federation.
• Categorizes graph FL into graph-level, subgraph-level, and node-level settings with corresponding typical tasks (graph classification, link prediction, node classification).
• Provides modular APIs and data loaders for easy experimentation, benchmarking, and deployment in cross-silo environments.

Experimental results

Research questions

• RQ1 How does federated learning affect GNN performance across graph-level, subgraph-level, and node-level FL settings?
• RQ2 What is the impact of non-IID data partitions on federated GNN accuracy compared to centralized training?
• RQ3 Which GNN architectures and FL algorithms are most robust or suffer the least accuracy degradation under graph FL?
• RQ4 What are the system efficiency and security characteristics of FedGraphNN on large-scale graph datasets?
• RQ5 What challenges remain in graph FL that require further methodological and benchmark improvements?

Key findings

• Federated GNNs often underperform centralized GNNs on larger non-IID graph datasets, while results on smaller datasets can be on par.
• The best centralized model does not necessarily translate to the best FL model, illustrating unique FL dynamics for graphs.
• GAT frequently shows larger accuracy gaps in graph-level FL, while some datasets (e.g., CIAO, CORA, PubMed) can outperform centralized training under subgraph- or node-level FL in certain cases.
• FedGraphNN demonstrates computational efficiency and security via LightSecAgg, achieving privacy with faster aggregation than some baselines.
• Training times range from minutes to about an hour depending on graph size, and secure aggregation maintains comparable privacy guarantees to SecAgg variants.