[Paper Review] Distance Encoding -- Design Provably More Powerful GNNs for Structural Representation Learning

Pan Li, Yanbang Wang+2 more|arXiv (Cornell University)|Aug 31, 2020

Advanced Graph Neural NetworksComputer Science47 references23 citations

TL;DR

This paper proposes Distance Encoding (DE), a general class of structural features that enhance GNNs by encoding distances from a target node set to all other nodes, enabling them to surpass the 1-Weisfeiler-Lehman test in expressive power. The method improves performance on node role prediction, link prediction, and triangle prediction, achieving up to 15% higher accuracy and AUC compared to standard GNNs and SOTA baselines.

ABSTRACT

Learning structural representations of node sets from graph-structured data is crucial for applications ranging from node-role discovery to link prediction and molecule classification. Graph Neural Networks (GNNs) have achieved great success in structural representation learning. However, most GNNs are limited by the 1-Weisfeiler-Lehman (WL) test and thus possible to generate identical representation for structures and graphs that are actually different. More powerful GNNs, proposed recently by mimicking higher-order-WL tests, only focus on entire-graph representations and cannot utilize sparsity of the graph structure to be computationally efficient. Here we propose a general class of structure-related features, termed Distance Encoding (DE), to assist GNNs in representing node sets with arbitrary sizes with strictly more expressive power than the 1-WL test. DE essentially captures the distance between the node set whose representation is to be learnt and each node in the graph, which includes important graph-related measures such as shortest-path-distance and generalized PageRank scores. We propose two general frameworks for GNNs to use DEs (1) as extra node attributes and (2) further as controllers of message aggregation in GNNs. Both frameworks may still utilize the sparse structure to keep scalability to process large graphs. In theory, we prove that these two frameworks can distinguish node sets embedded in almost all regular graphs where traditional GNNs always fail. We also rigorously analyze their limitations. Empirically, we evaluate these two frameworks on node structural roles prediction, link prediction and triangle prediction over six real networks. The results show that our models outperform GNNs without DEs by up-to 15% improvement in average accuracy and AUC. Our models also significantly outperform other SOTA baselines particularly designed for those tasks.

Motivation & Objective

• Address the limitation of standard GNNs, which are restricted by the 1-Weisfeiler-Lehman (1-WL) test and fail to distinguish structurally different graphs.
• Develop a general, scalable method to enhance GNNs with provably higher expressive power for structural representation learning.
• Enable GNNs to effectively leverage graph sparsity while learning representations of arbitrary-sized node sets.
• Design a feature engineering approach that captures global structural context via distance measures without sacrificing computational efficiency.

Proposed method

• Introduce Distance Encoding (DE) as a set of structure-related features that encode the distance from each node in the graph to a target node set, using measures like shortest-path distance and generalized PageRank scores.
• Integrate DE as extra node attributes in GNNs to enrich node representations with global structural context.
• Extend the GNN message-passing mechanism by using DE as controllers of message aggregation, enabling dynamic, distance-aware information flow.
• Ensure computational efficiency by preserving the sparse structure of the graph, allowing scalability to large graphs.
• Theoretical analysis proves that both DE-integrated frameworks can distinguish node sets embedded in almost all regular graphs where 1-WL-based GNNs fail.
• Formalize the expressive power of the proposed frameworks using graph isomorphism theory, showing they exceed the 1-WL test in distinguishing non-isomorphic graphs.

Experimental results

Research questions

• RQ1 Can Distance Encoding provide a provably more expressive alternative to 1-WL-based GNNs for structural representation learning?
• RQ2 How can DE be integrated into GNNs in a way that maintains computational efficiency on large, sparse graphs?
• RQ3 To what extent do DE-enhanced GNNs improve performance on node role prediction, link prediction, and triangle prediction tasks?
• RQ4 What are the theoretical limitations of the proposed DE-based frameworks in distinguishing graph structures?

Key findings

• The proposed DE-based GNN frameworks achieve up to 15% higher average accuracy and AUC on node structural role prediction, link prediction, and triangle prediction tasks across six real-world networks.
• DE-enhanced GNNs significantly outperform standard GNNs that lack structural encoding, demonstrating the critical role of distance-aware features in improving model expressivity.
• The models surpass other state-of-the-art baselines specifically designed for node role, link, and triangle prediction, particularly in challenging structural generalization tasks.
• Theoretical analysis confirms that both frameworks can distinguish node sets in almost all regular graphs where 1-WL-based GNNs fail, proving their superior expressive power.
• The integration of DE as either node attributes or message aggregation controllers maintains scalability due to the preservation of graph sparsity.
• Empirical results show consistent performance gains across diverse graph types and tasks, validating the generalizability of the DE approach.