[Paper Review] Position-aware Graph Neural Networks

Jiaxuan You, Rex Ying+1 more|arXiv (Cornell University)|Jun 11, 2019

Advanced Graph Neural Networks88 citations

TL;DR

P-GNNs introduce anchor-set based distance embeddings to capture node positions within graphs, enabling inductive, scalable, and feature-aware position-aware representations that outperform traditional GNNs on various tasks.

ABSTRACT

Learning node embeddings that capture a node's position within the broader graph structure is crucial for many prediction tasks on graphs. However, existing Graph Neural Network (GNN) architectures have limited power in capturing the position/location of a given node with respect to all other nodes of the graph. Here we propose Position-aware Graph Neural Networks (P-GNNs), a new class of GNNs for computing position-aware node embeddings. P-GNN first samples sets of anchor nodes, computes the distance of a given target node to each anchor-set,and then learns a non-linear distance-weighted aggregation scheme over the anchor-sets. This way P-GNNs can capture positions/locations of nodes with respect to the anchor nodes. P-GNNs have several advantages: they are inductive, scalable,and can incorporate node feature information. We apply P-GNNs to multiple prediction tasks including link prediction and community detection. We show that P-GNNs consistently outperform state of the art GNNs, with up to 66% improvement in terms of the ROC AUC score.

Motivation & Objective

• Motivate the need for node embeddings that encode a node's position in the graph beyond local neighborhood structure.
• Propose a scalable, inductive GNN framework (P-GNN) that uses anchor-sets and distance-weighted aggregation to learn position-aware embeddings.
• Demonstrate that P-GNNs are more expressive than traditional GNNs and can be instantiated as a fast variant (P-GNN-Fast).
• Empirically evaluate P-GNNs on multiple datasets and tasks, showing substantial performance gains over state-of-the-art GNNs.

Proposed method

• Sample k random anchor-sets S_i per forward pass to capture node positions relative to anchors.
• Compute per-anchor-set messages using a distance-based similarity s(v,u) and node features, then aggregate via learnable functions Agg_M and Agg_S.
• Project the anchor-set message matrix M by a trainable weight vector w to form the position-aware embedding z_v.
• Stack multiple P-GNN layers to increase expressiveness, preserving position-awareness and creating structure-aware messages h_v for propagation.
• Ground anchor-set selection in Bourgain’s theorem to guarantee low-distortion distance preservation with k = O(log^2 n).
• Provide a fast variant P-GNN-Fast with approximate distance computations to maintain scalable complexity comparable to traditional GNNs.

Experimental results

Research questions

• RQ1 Can node position relative to a set of shared anchor-sets be learned to distinguish structurally isomorphic nodes?
• RQ2 Do anchor-set based distance embeddings improve predictive performance on graph tasks such as link prediction and community detection compared to standard GNNs?
• RQ3 How does the expressiveness of P-GNNs compare to existing GNN architectures, and can P-GNNs be instantiated in a scalable fast variant?
• RQ4 What is the impact of using node features in conjunction with position-aware embeddings on downstream tasks?

Key findings

• P-GNNs consistently outperform state-of-the-art GNN variants across datasets and tasks, with up to 66% ROC AUC improvement in reported results.
• Ablation shows that incorporating position information via anchor-sets provides a meaningful boost over purely structure-aware embeddings.
• P-GNN-Fast achieves comparable performance to full P-GNN while maintaining scalable complexity similar to traditional GNNs.
• Theoretical analysis shows P-GNNs generalize conventional GNNs and provide greater expressive power by leveraging joint distributions of node embeddings with shared anchors.
• Anchor-sets sized according to Bourgain’s guidance (O(log^2 n)) enable low-distortion distance preservation.