[Paper Review] Learning Convolutional Neural Networks for Graphs

Mathias Niepert, Mohamed M. Ahmed+1 more|arXiv (Cornell University)|May 17, 2016

Advanced Graph Neural Networks816 citations

TL;DR

Patchy-san builds a framework to apply CNNs to arbitrary graphs by selecting node sequences, extracting and normalizing fixed-size local neighborhoods, and learning with CNNs, achieving competitive results with graph kernels and scalable performance.

ABSTRACT

Numerous important problems can be framed as learning from graph data. We propose a framework for learning convolutional neural networks for arbitrary graphs. These graphs may be undirected, directed, and with both discrete and continuous node and edge attributes. Analogous to image-based convolutional networks that operate on locally connected regions of the input, we present a general approach to extracting locally connected regions from graphs. Using established benchmark data sets, we demonstrate that the learned feature representations are competitive with state of the art graph kernels and that their computation is highly efficient.

Motivation & Objective

• Motivate applying CNNs to a broad class of graphs beyond grid-structured data.
• Develop a framework (Patchy-san) to create local, fixed-size receptive fields on graphs with diverse attributes.
• Learn graph representations end-to-end using CNNs without handcrafted graph kernels.
• Demonstrate efficiency and competitive performance against state-of-the-art graph kernels on benchmark datasets.

Proposed method

• Define Patchy-san (Select-Assemble-Normalize) to process graphs without a fixed node correspondence.
• Use graph labeling procedures (e.g., Weisfeiler-Lehman) to order nodes and create normalized receptive fields of size k.
• Construct receptive fields via neighborhood assembly with BFS up to k nodes and normalize via a canonical labeling to map to a fixed vector space.
• Feed the sequence of patches through CNN components (convolutional and dense layers) to learn graph representations.
• Provide theoretical results: NP-hardness of optimal graph normalization and conditions under which Patchy-san can emulate standard CNNs on grids.
• Demonstrate scalability with linear-ish complexity in number of graphs and efficient patch generation.

Experimental results

Research questions

• RQ1 Can Patchy-san learn discriminative graph representations for classification and regression on unseen graphs without aligned node correspondences?
• RQ2 How well do CNN-based graph representations perform compared to traditional graph kernels on standard benchmarks?
• RQ3 What are the computational trade-offs in patch generation and normalization, and how does labeling choice affect performance?
• RQ4 Does Patchy-san generalize CNNs on images to arbitrary graph topologies while preserving theoretical connections?

Key findings

• Patchy-san achieves competitive accuracy with state-of-the-art graph kernels on multiple benchmark datasets.
• The method scales linearly with the number of graphs and remains efficient for large graphs due to parallelizable receptive-field computation.
• Using 1-WL based normalization, receptive fields can emulate the first CNN layer on grid graphs, establishing theoretical equivalence in certain cases.
• In experiments on graph classification, Patchy-san with receptive field sizes around 10 often yields best accuracy across datasets.
• Patchy-san supports both node and edge attributes and can integrate continuous features via simple preprocessing (e.g., normalized node degree).
• Visualization experiments show learned graph motifs via RBMs trained on patches, illustrating interpretable local structures learned by the model.