[論文レビュー] Non-convolutional Graph Neural Networks

Yuanqing Wang, Kyunghyun Cho|arXiv (Cornell University)|Jul 31, 2024

Advanced Graph Neural Networks被引用数 5

ひとこと要約

RUM は WL を超えた表現力を持つ畳み込みなしの GNN で、ランダムウォークと RNN を用いて意味論的特徴とトポロジー的特徴を統合し、WLより高い表現力を達成し、過度平滑化と過度潰しを緩和しつつ、GPU上でのスケーラビリティと高速性を維持します。

ABSTRACT

Rethink convolution-based graph neural networks (GNN) -- they characteristically suffer from limited expressiveness, over-smoothing, and over-squashing, and require specialized sparse kernels for efficient computation. Here, we design a simple graph learning module entirely free of convolution operators, coined random walk with unifying memory (RUM) neural network, where an RNN merges the topological and semantic graph features along the random walks terminating at each node. Relating the rich literature on RNN behavior and graph topology, we theoretically show and experimentally verify that RUM attenuates the aforementioned symptoms and is more expressive than the Weisfeiler-Lehman (WL) isomorphism test. On a variety of node- and graph-level classification and regression tasks, RUM not only achieves competitive performance, but is also robust, memory-efficient, scalable, and faster than the simplest convolutional GNNs.

研究の動機と目的

• Motivate revisiting graph neural networks to address limited expressiveness, over-smoothing, and over-squashing in convolution-based GNNs.
• Propose a convolution-free module that integrates topological and semantic walk information.
• theoretically and empirically show RUM's superior expressiveness and robustness over traditional GNNs.
• Demonstrate RUM's efficiency, scalability, and competitive or superior performance on node- and graph-level tasks.

提案手法

• Construct random walks terminating at each node and describe their topological environment via an anonymous experiment.
• Merge semantic walk embeddings and topological walk indices through a unifying memory function using φx, φu, and f.
• Aggregate node representations across sampled l-step walks with Monte Carlo weighting to obtain ψ(v) for node-level tasks.
• Optionally pool node representations to form global graph representations Ψ(𝒢) for graph-level tasks.
• Ensure permutation equivariance of the model and highlight constant parameter count w.r.t. neighborhood size due to RNN-based sequencing.

Figure 1: RUM can (in closed form), whereas the Weisfeiler-Lehman (WL) isomorphism test and WL-equivalent GNNs cannot , distinguish these two graphs— an illustration of Example 8.1 .

実験結果

リサーチクエスチョン

• RQ1 Can RUM distinguish non-isomorphic graphs beyond the capabilities of WL-based GNNs?
• RQ2 Does the convolution-free structure alleviate over-smoothing and over-squashing seen in traditional GNNs?
• RQ3 Is RUM faster and more robust than convolutional GNNs on standard benchmarks?
• RQ4 How well does RUM scale to large graphs and dense graphs?

主な発見

• RUM can distinguish non-isomorphic graphs with sufficiently long random walks, exceeding WL test capabilities.
• RUM mitigates over-smoothing by maintaining Dirichlet energy along walks and using non-contractive mappings.
• RUM reduces over-squashing and demonstrates favorable gradient behavior compared to convolutional GNNs.
• RUM delivers competitive or superior performance on node and graph tasks across benchmarks and is faster on GPUs than simple GCNs.
• RUM shows robustness to graph perturbations and scales effectively to large graphs, with limitations on very dense graphs.

Figure 2: RUM alleviates over-smoothing . Dirichlet energy ( $\mathcal{E}$ ) on Cora [ 45 ] graph plotted against $L$ , the number of steps or layers.