[Paper Review] Graph Neural Processes: Towards Bayesian Graph Neural Networks

Andrew N. Carr, David Wingate|arXiv (Cornell University)|Feb 26, 2019

Advanced Graph Neural Networks37 references4 citations

TL;DR

This paper introduces Graph Neural Processes (GNP), a Bayesian deep learning framework that extends Conditional Neural Processes to graph-structured data, enabling uncertainty-aware edge imputation and generalization across variable-sized graphs. GNP achieves state-of-the-art performance on multiple graph benchmark datasets, outperforming random forests and baselines in precision, F1-score, and recall, particularly on imbalanced data.

ABSTRACT

We introduce Graph Neural Processes (GNP), inspired by the recent work in conditional and latent neural processes. A Graph Neural Process is defined as a Conditional Neural Process that operates on arbitrary graph data. It takes features of sparsely observed context points as input, and outputs a distribution over target points. We demonstrate graph neural processes in edge imputation and discuss benefits and drawbacks of the method for other application areas. One major benefit of GNPs is the ability to quantify uncertainty in deep learning on graph structures. An additional benefit of this method is the ability to extend graph neural networks to inputs of dynamic sized graphs.

Motivation & Objective

• To develop a Bayesian deep learning framework for graph-structured data that quantifies uncertainty in predictions.
• To extend the Conditional Neural Process (CNP) framework to operate on arbitrary graph inputs, enabling invariance to context point ordering.
• To enable generalization across graphs of varying sizes by learning a shared, global representation from local node and edge features.
• To demonstrate the utility of GNPs in edge imputation tasks, particularly in scenarios with noisy or missing edge values.
• To establish a foundation for future research in Bayesian graph neural networks and relational reasoning.

Proposed method

• GNP models a conditional distribution over target edge values given a set of observed (context) edge features, using a graph-structured input representation.
• It employs a graph neural network (GNN) encoder to embed context node and edge features into fixed-length representations, which are aggregated via a commutative operation (e.g., mean pooling) into a context vector $ r_C $.
• The context vector $ r_C $ is used as a global context to condition a decoder network that predicts the distribution over target edge values.
• The model is trained using maximum likelihood or cross-entropy loss, depending on the output type (continuous or categorical), and supports arbitrary sampling of context and target edges.
• The architecture is invariant to the ordering of context and target points, making it suitable for graph data with irregular structures.
• It leverages inductive biases from GNNs and the probabilistic inductive bias of CNPs to learn high-level abstractions across families of graphs.

Experimental results

Research questions

• RQ1 Can a neural process framework be successfully adapted to operate on graph-structured data while preserving invariance to context ordering?
• RQ2 Can Graph Neural Processes effectively model uncertainty in edge value predictions on graphs with missing or noisy data?
• RQ3 How well do GNPs generalize across graphs of varying sizes and structures, particularly in low-data or imbalanced settings?
• RQ4 What performance gains does GNP achieve over traditional baselines like random forests in edge imputation tasks?
• RQ5 In what application domains can GNPs provide meaningful improvements through uncertainty estimation and relational reasoning?

Key findings

• On the Tox21 dataset, GNP outperforms all baselines by approximately 20% in precision, 12% in F1-score, and 10% in recall, demonstrating strong generalization on imbalanced data.
• On the PTC family of datasets, GNP improves precision by 10–15% and F1-score by 3–10% over random forests, indicating superior performance in semi-supervised edge prediction.
• On the MUTAG dataset, GNP significantly outperforms random forests and naive baselines, achieving higher accuracy in classifying mutagenic compounds.
• Despite high class imbalance (e.g., 90% accuracy from predicting the majority class on bzr), GNP achieves the best F1 and recall on two out of four datasets, showing robustness to imbalance.
• GNP demonstrates strong performance across diverse graph types, including molecular compounds, traffic networks, and knowledge graphs, validating its broad applicability.
• The model successfully learns global representations across families of graphs, enabling effective few-shot and dynamic graph reasoning, as suggested by its performance on diverse benchmarks.