[Paper Review] Principal Neighbourhood Aggregation for Graph Nets

Gabriele Corso, Luca Cavalleri+3 more|arXiv (Cornell University)|Apr 12, 2020

Advanced Graph Theory ResearchComputer Science47 references244 citations

TL;DR

The paper introduces Principal Neighbourhood Aggregation (PNA), a GNN layer combining multiple aggregators with degree-scalers to handle continuous features, and demonstrates its superior performance on synthetic multi-task benchmarks and real-world datasets.

ABSTRACT

Graph Neural Networks (GNNs) have been shown to be effective models for different predictive tasks on graph-structured data. Recent work on their expressive power has focused on isomorphism tasks and countable feature spaces. We extend this theoretical framework to include continuous features - which occur regularly in real-world input domains and within the hidden layers of GNNs - and we demonstrate the requirement for multiple aggregation functions in this context. Accordingly, we propose Principal Neighbourhood Aggregation (PNA), a novel architecture combining multiple aggregators with degree-scalers (which generalize the sum aggregator). Finally, we compare the capacity of different models to capture and exploit the graph structure via a novel benchmark containing multiple tasks taken from classical graph theory, alongside existing benchmarks from real-world domains, all of which demonstrate the strength of our model. With this work, we hope to steer some of the GNN research towards new aggregation methods which we believe are essential in the search for powerful and robust models.

Motivation & Objective

• Motivate the need for richer neighborhood aggregation in GNNs with continuous features.
• Propose the Principal Neighbourhood Aggregation (PNA) architecture to couple multiple aggregators with degree-based scalers.
• Theoretically establish the necessity of multiple aggregators for injectivity in continuous feature spaces.
• Empirically evaluate PNA on a multi-task synthetic benchmark and real-world datasets, comparing against standard GNN layers.

Proposed method

• Define continuous-feature injectivity limits for neighborhood multisets using aggregators.
• Introduce degree-scalers, including a logarithmic scaler, to modulate messages by node degree (d).
• Combine four aggregators (mean, max, min, standard deviation) with three degree-scalers via a tensor product to form the PNA operator.
• Embed PNA in an encode-process-decode GNN architecture with shared parameters across layers and a variable depth.
• Develop a multi-task synthetic benchmark inspired by classical graph problems to test expressive power and generalization.
• Evaluate PNA on real-world graph datasets (ZINC, MolHIV, CIFAR10, MNIST) following established training protocols for fair comparison.

Experimental results

Research questions

• RQ1 Does using multiple aggregators improve GNN expressiveness with continuous features compared to single-aggregator models?
• RQ2 How do degree-scalers affect the discrimination of neighborhoods of different sizes in GNNs?
• RQ3 Can PNA outperform existing GNN layers on synthetic multi-task benchmarks and real-world graph datasets?
• RQ4 What is the impact of PNA on extrapolation to larger graphs and varying graph domains?

Key findings

• PNA consistently outperforms state-of-the-art models on the multi-task artificial benchmark.
• PNA shows stronger performance across all architecture layouts and graph types tested.
• In real-world chemical benchmarks, PNA (with scalers) improves performance, while the version without scalers struggles on neighborhood-size discrimination.
• Increasing latent dimensions in baseline models does not bridge the gap to PNA performance.
• PNA maintains superior performance on larger graphs in extrapolation tests, though some feature explosions can occur with other models.