[Paper Review] pathGCN: Learning General Graph Spatial Operators from Paths

Moshe Eliasof, Eldad Haber+1 more|arXiv (Cornell University)|Jul 15, 2022

Advanced Graph Neural Networks4 citations

TL;DR

This paper proposes pathGCN, a novel method to learn expressive graph spatial operators from random paths instead of relying on a fixed graph Laplacian. By training spatial kernels end-to-end via path-based aggregation, pathGCN enables deeper, more flexible GCNs that avoid over-smoothing and achieve state-of-the-art performance on node classification benchmarks.

ABSTRACT

Graph Convolutional Networks (GCNs), similarly to Convolutional Neural Networks (CNNs), are typically based on two main operations - spatial and point-wise convolutions. In the context of GCNs, differently from CNNs, a pre-determined spatial operator based on the graph Laplacian is often chosen, allowing only the point-wise operations to be learnt. However, learning a meaningful spatial operator is critical for developing more expressive GCNs for improved performance. In this paper we propose pathGCN, a novel approach to learn the spatial operator from random paths on the graph. We analyze the convergence of our method and its difference from existing GCNs. Furthermore, we discuss several options of combining our learnt spatial operator with point-wise convolutions. Our extensive experiments on numerous datasets suggest that by properly learning both the spatial and point-wise convolutions, phenomena like over-smoothing can be inherently avoided, and new state-of-the-art performance is achieved.

Motivation & Objective

• Address the limitation of fixed spatial operators in GCNs, which restrict model expressiveness and cause over-smoothing in deep architectures.
• Overcome the reliance on the graph Laplacian in standard GCNs by learning spatial operators directly from data.
• Enable GCNs to achieve CNN-like flexibility by learning spatial kernels with variable aperture and adaptive coefficients.
• Prevent over-smoothing in deep GCNs by learning dynamic spatial operators instead of using a fixed Laplacian.
• Improve performance across diverse graph benchmarks by jointly learning spatial and point-wise convolutions.

Proposed method

• Define spatial operators based on random walks (paths) on the graph, where each path aggregates features from neighboring nodes.
• Learn the coefficients of the spatial operator end-to-end via stochastic optimization, allowing the network to adapt the kernel to the data and task.
• Formulate pathGCN with three variants: global (shared across layers/channels), per-layer (shared across channels), and depth-wise (per-channel and per-layer), emulating CNN-style learning.
• Use a stochastic path training policy to sample paths during backpropagation, enabling efficient and differentiable learning of spatial operations.
• Integrate the learned spatial operators with learnable 1×1 point-wise convolutions to form a full GCN block.
• Apply symmetric normalization and self-loops to the adjacency matrix to stabilize training and improve generalization.

Experimental results

Research questions

• RQ1 Can learning spatial operators from random paths improve the expressiveness and performance of GCNs compared to fixed Laplacian-based operators?
• RQ2 Does path-based learning of spatial kernels prevent over-smoothing in deep GCNs, which commonly occurs with repeated application of the Laplacian?
• RQ3 How do different formulations of pathGCN (global, per-layer, depth-wise) affect model performance and generalization?
• RQ4 What is the impact of path length and number of sampled paths on the final accuracy and robustness of the learned spatial operators?
• RQ5 Can the learned spatial kernels learn diverse functions such as smoothing, edge detection, or sharpening, similar to CNN filters?

Key findings

• pathGCN achieves state-of-the-art accuracy on Cora, Citeseer, and Pubmed datasets, with 85.8% accuracy on Cora using 64 layers, outperforming prior methods.
• The depth-wise variant (pathGCN) consistently outperforms both global and per-layer variants, indicating that per-channel spatial learning enhances expressiveness.
• Over-smoothing is inherently avoided in pathGCN, even with deep architectures (up to 64 layers), due to the learned, non-uniform spatial operators.
• Performance improves with increasing path length up to k=7, after which accuracy stagnates or declines, indicating an optimal kernel size for feature aggregation.
• Using more paths (p > 20) leads to performance degradation, suggesting that too much stochasticity can hurt generalization, with optimal performance at p=10.
• Learnt spatial kernels exhibit diverse behaviors—such as smoothing and edge detection—demonstrating that pathGCN can learn task-specific, adaptive spatial operations.