[Paper Review] Can GCNs Go as Deep as CNNs

Guohao Li, Matthias Müller+2 more|arXiv (Cornell University)|Apr 7, 2019

Advanced Graph Neural Networks36 citations

TL;DR

This paper proposes a deep Graph Convolutional Network (GCN) framework inspired by residual and dense connections from CNNs, combined with dilated convolutions, enabling training of very deep GCNs—demonstrated by a 56-layer model that achieves a 3.7% mIoU improvement over state-of-the-art on point cloud semantic segmentation.

ABSTRACT

Convolutional Neural Networks (CNNs) achieve impressive performance in a wide variety of fields. Their success benefited from a massive boost when very deep CNN models were able to be reliably trained. Despite their merits, CNNs fail to properly address problems with non-Euclidean data. To overcome this challenge, Graph Convolutional Networks (GCNs) build graphs to represent non-Euclidean data, borrow concepts from CNNs, and apply them in training. GCNs show promising results, but they are usually limited to very shallow models due to the vanishing gradient problem. As a result, most state-of-the-art GCN models are no deeper than 3 or 4 layers. In this work, we present new ways to successfully train very deep GCNs. We do this by borrowing concepts from CNNs, specifically residual/dense connections and dilated convolutions, and adapting them to GCN architectures. Extensive experiments show the positive effect of these deep GCN frameworks. Finally, we use these new concepts to build a very deep 56-layer GCN, and show how it significantly boosts performance (+3.7% mIoU over state-of-the-art) in the task of point cloud semantic segmentation. We believe that the community can greatly benefit from this work, as it opens up many opportunities for advancing GCN-based research.

Motivation & Objective

• To address the vanishing gradient problem that limits GCN depth in existing models.
• To enable training of very deep GCN architectures comparable to deep CNNs.
• To improve performance on non-Euclidean data, particularly in point cloud semantic segmentation.
• To adapt successful CNN techniques—residual and dense connections, dilated convolutions—into GCN frameworks.

Proposed method

• Adapts residual and dense skip connections from CNNs to GCNs to mitigate vanishing gradients in deep architectures.
• Introduces dilated convolutions in GCNs to expand感受 receptive fields without increasing parameters or layers.
• Designs a deep GCN architecture using residual connections that allow training of up to 56 layers.
• Employs graph convolutional layers with learnable filters that aggregate features from neighboring nodes in a graph.
• Uses identity shortcuts to preserve gradient flow across layers, improving training stability.
• Combines multi-scale感受 via dilated convolutions to enhance feature representation in deep GCNs.

Experimental results

Research questions

• RQ1 Can GCNs be trained as deeply as CNNs without suffering from vanishing gradients?
• RQ2 How do residual and dense connections improve training depth in GCN architectures?
• RQ3 What is the impact of dilated convolutions on receptive field expansion in GCNs?
• RQ4 Can a 56-layer GCN achieve state-of-the-art performance on point cloud semantic segmentation?
• RQ5 What performance gains are achievable by combining residual, dense, and dilated mechanisms in GCNs?

Key findings

• The proposed deep GCN framework successfully trains a 56-layer model, significantly deeper than typical GCNs which are limited to 3–4 layers.
• The 56-layer GCN achieves a 3.7% mIoU improvement over state-of-the-art methods on point cloud semantic segmentation.
• Residual and dense connections effectively alleviate the vanishing gradient problem in deep GCNs.
• Dilated convolutions enhance feature representation by expanding the receptive field without increasing model depth.
• The combination of residual, dense, and dilated mechanisms leads to consistent performance gains across experiments.
• The framework demonstrates that GCNs can achieve performance comparable to deep CNNs in non-Euclidean data tasks.