[Paper Review] Subgroup Generalization and Fairness of Graph Neural Networks

Jiaqi Ma, Junwei Deng+1 more|arXiv (Cornell University)|Jun 29, 2021

Advanced Graph Neural NetworksComputer Science45 references27 citations

TL;DR

The paper develops a novel PAC-Bayesian analysis for GNNs in non-IID node-level semi-supervised learning, derives subgroup generalization bounds, and demonstrates theoretical and empirical accuracy disparities across test node subgroups related to distance from training data.

ABSTRACT

Despite enormous successful applications of graph neural networks (GNNs), theoretical understanding of their generalization ability, especially for node-level tasks where data are not independent and identically-distributed (IID), has been sparse. The theoretical investigation of the generalization performance is beneficial for understanding fundamental issues (such as fairness) of GNN models and designing better learning methods. In this paper, we present a novel PAC-Bayesian analysis for GNNs under a non-IID semi-supervised learning setup. Moreover, we analyze the generalization performances on different subgroups of unlabeled nodes, which allows us to further study an accuracy-(dis)parity-style (un)fairness of GNNs from a theoretical perspective. Under reasonable assumptions, we demonstrate that the distance between a test subgroup and the training set can be a key factor affecting the GNN performance on that subgroup, which calls special attention to the training node selection for fair learning. Experiments across multiple GNN models and datasets support our theoretical results.

Motivation & Objective

• Provide a PAC-Bayesian framework for GNN generalization under non-IID node-level semi-supervised learning.
• Derive subgroup-specific generalization bounds that depend on the distance between test and training node features.
• Theoretically and empirically investigate accuracy disparities (fairness) across test subgroups defined by distance to training data.
• Highlight how training node selection and graph structure influence fairness in GNNs.

Proposed method

• Formulate node-level semi-supervised learning on graphs with fixed X,G and label randomness conditioned on aggregated features Z=g(X,G).
• Develop PAC-Bayesian bounds for both stochastic and deterministic classifiers across subgroups, introducing the expected loss discrepancy D^gamma_{m,m'}(P;λ).
• Specialize the bounds to GNNs where h_i(X,G)=f(g_i(X,G);W1,...,WL) and derive a subgroup generalization bound that includes epsilon_m, the distance of subgroup V_m to the training set V0.
• Impose smoothness (Assumption 1) on label distributions conditioned on aggregated features and model- and data-dependent assumptions (Assumptions 2-4) to bound the discrepancy term D^gamma_{m,0}(P;λ).
• Present Theorem 3 giving a subgroup generalization bound for GNNs showing dependence on epsilon_m, network width b, layer norms, and training size N0.

Experimental results

Research questions

• RQ1 How does non-IID node-level data affect GNN generalization in semi-supervised settings?
• RQ2 Can we bound subgroup-specific generalization errors for GNNs and understand factors driving disparities across subgroups?
• RQ3 Does the distance between test subgroups and the training set predict accuracy/disparity, informing fairness of GNNs?
• RQ4 How do training-data selection and graph structure influence subgroup fairness in GNN predictions?

Key findings

• A PAC-Bayesian generalization bound for non-IID node-level GNNs is developed, linking subgroup generalization to an expected loss discrepancy between training and test subgroups.
• The subgroup generalization bound for GNNs shows the test error on a subgroup grows with the aggregated-feature distance epsilon_m to the training set.
• The bound indicates potential unfairness: subgroups farther from the training data have weaker generalization guarantees and potentially lower accuracy.
• Empirical results across GCN, GAT, SGC, and APPNP on Cora, Citeseer, and PubMed show significant accuracy disparities across subgroups defined by distance to training data and geodesic distance, validating the theory.
• Biased training node selection (centrality-based) exacerbates disparities, highlighting the impact of training data choice on fairness.