[Paper Review] Tree Decomposed Graph Neural Network

Yu Wang, Tyler Derr|arXiv (Cornell University)|Aug 25, 2021

Advanced Graph Neural Networks44 references4 citations

TL;DR

This paper proposes Tree Decomposed Graph Neural Network (TDGNN), a novel GNN architecture that mitigates feature smoothing between different layers by using tree decomposition to disentangle multi-layer neighborhood information. By modeling multi-hop dependencies via graph diffusion within the tree structure, TDGNN enables deeper, more flexible representation learning and achieves state-of-the-art performance on both homophily and heterophily networks across multiple node classification benchmarks.

ABSTRACT

Graph Neural Networks (GNNs) have achieved significant success in learning better representations by performing feature propagation and transformation iteratively to leverage neighborhood information. Nevertheless, iterative propagation restricts the information of higher-layer neighborhoods to be transported through and fused with the lower-layer neighborhoods', which unavoidably results in feature smoothing between neighborhoods in different layers and can thus compromise the performance, especially on heterophily networks. Furthermore, most deep GNNs only recognize the importance of higher-layer neighborhoods while yet to fully explore the importance of multi-hop dependency within the context of different layer neighborhoods in learning better representations. In this work, we first theoretically analyze the feature smoothing between neighborhoods in different layers and empirically demonstrate the variance of the homophily level across neighborhoods at different layers. Motivated by these analyses, we further propose a tree decomposition method to disentangle neighborhoods in different layers to alleviate feature smoothing among these layers. Moreover, we characterize the multi-hop dependency via graph diffusion within our tree decomposition formulation to construct Tree Decomposed Graph Neural Network (TDGNN), which can flexibly incorporate information from large receptive fields and aggregate this information utilizing the multi-hop dependency. Comprehensive experiments demonstrate the superior performance of TDGNN on both homophily and heterophily networks under a variety of node classification settings. Extensive parameter analysis highlights the ability of TDGNN to prevent over-smoothing and incorporate features from shallow layers with deeper multi-hop dependencies, which provides new insights towards deeper graph neural networks. Code of TDGNN: http://github.com/YuWVandy/TDGNN

Motivation & Objective

• To address the over-smoothing problem in deep GNNs caused by iterative feature propagation across layers.
• To analyze and mitigate feature smoothing between neighborhoods in different layers, which compromises performance—especially in heterophily graphs.
• To explore the importance of multi-hop dependency in node representation learning and its connection to graph diffusion.
• To design a flexible, deep GNN architecture that decouples neighborhood aggregation across layers and enables selective, depth-optimized information fusion.
• To demonstrate superior performance on diverse real-world graphs, including challenging heterophily networks.

Proposed method

• Proposes a tree decomposition method to disentangle neighborhoods across different layers, allowing independent processing of each layer's receptive field.
• Models multi-hop dependencies using graph diffusion within the tree decomposition framework to preserve long-range structural information.
• Introduces two variants: TDGNN-s (single-path) and TDGNN-w (wide-path), enabling flexible aggregation of representations from different layers.
• Employs a decoupled architecture where feature transformation and propagation are separated, reducing entanglement-induced smoothing.
• Uses personalized PageRank-based diffusion to compute node importance and propagate information along tree-structured paths.
• Applies adaptive aggregation to combine representations from shallow and deep layers, enhancing feature richness without over-smoothing.

Experimental results

Research questions

• RQ1 How does feature smoothing between neighborhoods in different layers affect GNN performance, especially in heterophily networks?
• RQ2 To what extent can tree decomposition reduce inter-layer feature smoothing in deep GNNs?
• RQ3 What is the role of multi-hop dependency in improving node representation learning, and how can it be effectively modeled?
• RQ4 Can a tree-based decomposition framework enable more flexible and deeper GNN architectures that outperform existing SOTA models?
• RQ5 How does the integration of graph diffusion within tree decomposition enhance representation learning across diverse graph types?

Key findings

• TDGNN significantly outperforms SOTA baselines, including GCNII and DAGNN, on both homophily and heterophily networks across multiple node classification benchmarks.
• The model achieves state-of-the-art performance on citation networks (PubMed, Cora, Citeseer) and heterophily datasets (Photo, Computers from Amazon, Reddit), demonstrating robustness across graph types.
• Extensive ablation studies confirm that tree decomposition effectively reduces feature smoothing between layers, preserving discriminative information.
• The incorporation of multi-hop dependency via graph diffusion improves performance, especially in deeper architectures, validating its theoretical and empirical significance.
• TDGNN-w, the wide-path variant, shows greater performance gains on heterophily datasets, indicating the benefit of wider, multi-layered information fusion.
• Parameter analysis reveals that TDGNN maintains low over-smoothing even with increased depth, unlike standard GNNs that degrade after 2–3 layers.