[Paper Review] Boosting Graph Structure Learning with Dummy Nodes

Xin Liu, Jay J. Cheng+2 more|arXiv (Cornell University)|Jun 17, 2022

Advanced Graph Neural Networks4 citations

TL;DR

This paper proposes a novel method to enhance graph structure learning by introducing a dummy node connected to all existing vertices without altering original graph properties. The dummy node enables a lossless, monomorphic edge-to-vertex transformation and its inverse, preserving local and global structures, which boosts performance in graph classification and subgraph isomorphism tasks across graph kernels and GNNs with linear computational cost.

ABSTRACT

With the development of graph kernels and graph representation learning, many superior methods have been proposed to handle scalability and oversmoothing issues on graph structure learning. However, most of those strategies are designed based on practical experience rather than theoretical analysis. In this paper, we use a particular dummy node connecting to all existing vertices without affecting original vertex and edge properties. We further prove that such the dummy node can help build an efficient monomorphic edge-to-vertex transform and an epimorphic inverse to recover the original graph back. It also indicates that adding dummy nodes can preserve local and global structures for better graph representation learning. We extend graph kernels and graph neural networks with dummy nodes and conduct experiments on graph classification and subgraph isomorphism matching tasks. Empirical results demonstrate that taking graphs with dummy nodes as input significantly boosts graph structure learning, and using their edge-to-vertex graphs can also achieve similar results. We also discuss the gain of expressive power from the dummy in neural networks.

Motivation & Objective

• To address the limitations of existing graph structure learning methods that rely on heuristic or empirically tuned strategies rather than theoretical grounding.
• To investigate whether a specially designed dummy node can enable a lossless, structure-preserving transformation from edges to vertices in graphs.
• To extend graph kernels and graph neural networks using dummy nodes to improve their ability to capture complex subgraph structures.
• To empirically validate the effectiveness of the proposed method on graph classification and subgraph isomorphism matching tasks.
• To analyze the expressive power gains of message-passing neural networks and transformers when augmented with dummy nodes.

Proposed method

• Introduce a dummy node connected to all existing vertices, preserving original vertex and edge properties.
• Design an efficient monomorphic edge-to-vertex transform $ L_{\Phi} $ that maps edges to vertices while maintaining structural integrity.
• Construct an epimorphic inverse transform $ L_{\Phi}^{-1} $ to recover the original graph from the transformed graph.
• Apply the edge-to-vertex transformation to extend vertex-centric graph kernels and GNNs, enabling them to implicitly learn edge-based substructures.
• Integrate dummy nodes into GNNs and graph kernels, maintaining linear time complexity relative to the number of edges.
• Utilize the transformed graph $ \mathcal{H}_{\Phi} $ as input to GNNs and graph kernels, enabling improved performance on structure-sensitive tasks.

Experimental results

Research questions

• RQ1 Can a dummy node connected to all vertices enable a lossless transformation from edges to vertices while preserving graph structure?
• RQ2 Does the proposed edge-to-vertex transform and its inverse maintain the original graph’s topological and substructural properties?
• RQ3 To what extent does incorporating dummy nodes improve the performance of graph kernels and GNNs on graph classification and subgraph isomorphism tasks?
• RQ4 How does the expressive power of message-passing neural networks change when augmented with dummy nodes?
• RQ5 Can transformers with CLS tokens (analogous to dummy nodes) achieve higher expressive power than standard GNNs?

Key findings

• The proposed edge-to-vertex transform $ L_{\Phi} $ is monomorphic and lossless, with a well-defined inverse $ L_{\Phi}^{-1} $ that recovers the original graph.
• Graphs with dummy nodes significantly improve performance on graph classification and subgraph isomorphism matching, with HGT achieving a 49.59% relative error reduction in GED.
• DMPNN-LRP achieves the lowest errors on homogeneous data, but performance degrades on heterogeneous graphs due to challenges in edge representation learning via line graphs.
• The combination of reversed edges and dummy nodes enhances performance, particularly in RGIN, though the primary boost in DMPNN and DMPNN-LRP comes from reversed edges.
• MPNNs with $ \mathcal{H}_{\Phi} $ achieve expressive power beyond 2-WL, surpassing standard GNNs that are limited to 1-WL or 2-WL expressiveness.
• Transformers with CLS tokens (analogous to dummy nodes) are expected to surpass 2-WL in expressive power, with 12-layer models capable of capturing patterns up to size 12,288.