[Paper Review] Dissecting the Diffusion Process in Linear Graph Convolutional Networks

Yifei Wang, Yisen Wang+2 more|arXiv (Cornell University)|Feb 22, 2021

Advanced Graph Neural NetworksComputer Science33 references17 citations

TL;DR

This paper proposes Decoupled Graph Convolution (DGC), a linear Graph Convolutional Network that decouples terminal time $T$ from the number of propagation steps $K$, enabling fine-grained approximation of continuous graph diffusion. By fixing $T$ and increasing $K$, DGC avoids over-smoothing and achieves state-of-the-art performance among linear GCNs, outperforming SGC and rivaling non-linear GCNs while remaining computationally efficient.

ABSTRACT

Graph Convolutional Networks (GCNs) have attracted more and more attentions in recent years. A typical GCN layer consists of a linear feature propagation step and a nonlinear transformation step. Recent works show that a linear GCN can achieve comparable performance to the original non-linear GCN while being much more computationally efficient. In this paper, we dissect the feature propagation steps of linear GCNs from a perspective of continuous graph diffusion, and analyze why linear GCNs fail to benefit from more propagation steps. Following that, we propose Decoupled Graph Convolution (DGC) that decouples the terminal time and the feature propagation steps, making it more flexible and capable of exploiting a very large number of feature propagation steps. Experiments demonstrate that our proposed DGC improves linear GCNs by a large margin and makes them competitive with many modern variants of non-linear GCNs.

Motivation & Objective

• To investigate why linear GCNs like SGC fail to benefit from more propagation steps despite their computational efficiency.
• To identify the root cause of performance degradation in SGC as stemming from a fixed, coarse finite difference step size $\Delta t = 1$ leading to large numerical error and over-smoothing.
• To propose a method that decouples terminal time $T$ and propagation steps $K$, enabling flexible control over the diffusion process.
• To demonstrate that a properly designed linear GCN can match or exceed the performance of modern non-linear GCN variants.
• To establish DGC as a strong, efficient, and theoretically grounded baseline for future GCN research.

Proposed method

• DGC decouples the terminal time $T$ from the number of propagation steps $K$, allowing $T$ to be chosen continuously for optimal smoothing trade-off.
• The method uses a continuous-time diffusion framework, modeling graph convolution as a solution to a diffusion PDE, with $T$ representing the total diffusion time.
• Feature propagation is approximated via a fine-grained finite difference scheme with increasing $K$, reducing numerical error compared to SGC’s fixed $\Delta t = 1$.
• DGC employs the Euler method for numerical integration, which is sufficient and efficient, and outperforms higher-order schemes like Runge-Kutta in practice.
• Two Laplacian formulations are evaluated: $\mathbf{L} = \mathbf{I} - \mathbf{S}$ (with self-loops) and $\mathbf{L}_{\rm sym} = \mathbf{I} - \mathbf{S}_{\rm sym}$ (symmetric), with the former showing superior performance.
• The optimal terminal time $T^*$ is found to be dataset-specific and intrinsic, with $T^* = 6.0$ on Pubmed and $T^* = 2.7$ on Reddit.

Experimental results

Research questions

• RQ1 Why do linear GCNs like SGC degrade in performance with more propagation steps, despite their simplicity and efficiency?
• RQ2 How does the fixed step size $\Delta t = 1$ in SGC's finite difference approximation affect the accuracy and stability of the diffusion process?
• RQ3 Can decoupling the terminal time $T$ from the number of steps $K$ enable better approximation of continuous graph diffusion and mitigate over-smoothing?
• RQ4 What is the optimal terminal time $T^*$ for real-world graph datasets, and is it consistent across different Laplacian formulations?
• RQ5 Can a linear GCN with proper design outperform modern non-linear GCN variants in terms of accuracy and efficiency?

Key findings

• DGC significantly improves over SGC, achieving 80.3% test accuracy on Pubmed and 95.8% on Reddit, outperforming SGC and matching or exceeding many non-linear GCN variants.
• The optimal terminal time $T^*$ is found to be 6.0 for Pubmed and 2.7 for Reddit, indicating that $T^*$ is an intrinsic property of the dataset.
• Using the augmented Laplacian $\mathbf{L} = \mathbf{I} - \mathbf{S}$ (with self-loops) yields higher accuracy than $\mathbf{L}_{\rm sym}$ and requires fewer steps to converge.
• Increasing $K$ while fixing $T$ improves performance without over-smoothing, demonstrating that DGC avoids the fundamental flaw of SGC.
• DGC with $K=100$ achieves 225.0 ms total training time on Pubmed (vs. 65.3 ms for SGC), but with significantly higher accuracy, and remains over 100× faster than non-linear GCNs.
• The Euler method is sufficient for DGC, as higher-order schemes like Runge-Kutta show diminishing returns with increasing $K$, confirming the efficiency of the proposed approach.