[Paper Review] Graph Capsule Convolutional Neural Networks

Saurabh Verma, Zhili Zhang|arXiv (Cornell University)|May 21, 2018

Advanced Graph Neural NetworksComputer Science32 references95 citations

TL;DR

This paper introduces Graph Capsule CNNs (GCAPS-CNN) that use capsule-based outputs and a covariance-based permutation-invariant layer to improve graph classification, achieving state-of-the-art results on several benchmarks.

ABSTRACT

Graph Convolutional Neural Networks (GCNNs) are the most recent exciting advancement in deep learning field and their applications are quickly spreading in multi-cross-domains including bioinformatics, chemoinformatics, social networks, natural language processing and computer vision. In this paper, we expose and tackle some of the basic weaknesses of a GCNN model with a capsule idea presented in \cite{hinton2011transforming} and propose our Graph Capsule Network (GCAPS-CNN) model. In addition, we design our GCAPS-CNN model to solve especially graph classification problem which current GCNN models find challenging. Through extensive experiments, we show that our proposed Graph Capsule Network can significantly outperforms both the existing state-of-art deep learning methods and graph kernels on graph classification benchmark datasets.

Motivation & Objective

• Motivate and address weaknesses of standard GCNNs for graph classification.
• Introduce graph capsules to capture richer local information than scalar convolutions.
• Develop a fast, differentiable permutation-invariant layer based on covariance.
• Incorporate global graph features to exploit full graph structure.
• Demonstrate superior performance on multiple graph classification datasets.

Proposed method

• Replace scalar graph convolution outputs with vector-valued capsules encoding higher-order information.
• Use statistical moments or covariance-based instantiation parameters as capsule outputs.
• Enforce permutation invariance via a covariance-based invariant layer that aggregates without relying on node order.
• Integrate global graph features, such as Fgsd spectra, to augment node representations at each layer.
• Provide an end-to-end GCAPS-CNN architecture with layers, invariant pooling, and fully connected classifiers.
• Compare against deep learning methods and graph kernels on standard graph classification benchmarks.

Experimental results

Research questions

• RQ1 Can graph capsules preserve or recover information lost in standard GCNN aggregations?
• RQ2 Does a covariance-based permutation-invariant layer enable reliable graph classification across isomorphic graphs?
• RQ3 Do global graph features improve classification performance when integrated into GCNNs?
• RQ4 How does GCAPS-CNN compare to state-of-the-art graph kernels and deep graph models on benchmark datasets?

Key findings

• GCAPS-CNN outperforms existing state-of-the-art deep learning methods and graph kernels on several graph classification benchmarks.
• The proposed covariance-based invariant layer provides permutation invariance and efficient computation.
• Global features (e.g., Fgsd) can enhance performance by incorporating full-graph structure.
• GCAPS-CNN achieves best results on multiple datasets such as PROTEINS, NCI1, NCI109, and ENZYMES.
• The model demonstrates competitive or superior accuracy across both deep learning and kernel-based baselines.
• The architecture maintains end-to-end differentiability and is applicable to different GCNN variants.