[Paper Review] Adaptive Graph Convolutional Neural Networks

Ruoyu Li, Sheng Wang+2 more|arXiv (Cornell University)|Jan 10, 2018

Advanced Graph Neural NetworksComputer Science23 references95 citations

TL;DR

AGCN learns task-specific adaptive graphs by training a residual graph Laplacian and distance metric, enabling convolution on arbitrary graph structures with improved performance and convergence.

ABSTRACT

Graph Convolutional Neural Networks (Graph CNNs) are generalizations of classical CNNs to handle graph data such as molecular data, point could and social networks. Current filters in graph CNNs are built for fixed and shared graph structure. However, for most real data, the graph structures varies in both size and connectivity. The paper proposes a generalized and flexible graph CNN taking data of arbitrary graph structure as input. In that way a task-driven adaptive graph is learned for each graph data while training. To efficiently learn the graph, a distance metric learning is proposed. Extensive experiments on nine graph-structured datasets have demonstrated the superior performance improvement on both convergence speed and predictive accuracy.

Motivation & Objective

• Motivate graph CNNs to handle varying graph structures instead of fixed graphs.
• Propose a spectral graph convolution layer that learns an adaptive, sample-specific graph Laplacian.
• Introduce distance-metric learning to construct adaptive graphs.
• Incorporate feature-space re-parameterization and residual graph components to improve learning efficiency.
• Demonstrate superior performance across multiple graph-structured datasets including molecules and point clouds.

Proposed method

• Introduce SGC-LL: a spectral graph convolution layer with an adaptive Laplacian learned via a distance metric.
• Use Mahalanobis-like distance D(xi,xj) = sqrt((xi-xj)^T M (xi-xj)) with M = Wd Wd^T as trainable.
• Compute a residual graph Laplacian update L_res from learned metric and features, and form ilde L = L + alpha L_res.
• Represent spectral filter g_theta(L) as a polynomial in updated Laplacian using Chebyshev expansion for efficiency.
• Apply a feature-space transform Y = U g_theta(L) U^T X with trainable W and b to embed intra- and inter-vertex features.
• Adopt per-layer residual graph updates to allow batch training over graphs with differing topology and size.

Experimental results

Research questions

• RQ1 Can a graph CNN handle data with arbitrary graph structure and size without losing information due to fixed graphs?
• RQ2 Does learning a per-sample adaptive graph Laplacian improve predictive accuracy and convergence speed?
• RQ3 Can distance-metric learning effectively customize graph topology for a given task?
• RQ4 Does re-parameterization of vertex features enhance graph convolution performance?
• RQ5 How does the proposed AGCN perform on molecular, point-cloud, and multi-task toxicity datasets compared to existing graph CNNs?

Key findings

• AGCN outperforms state-of-the-art graph CNNs on multiple datasets, showing faster convergence and better predictive accuracy.
• Learning a residual Laplacian L_res per sample yields graph topologies that better serve the prediction task and can introduce new edges beyond intrinsic graphs.
• Distance-metric learning with a low parameter cost (O(d^2) or O(d)) makes topology updates computationally efficient and graph-size independent.
• The model supports training on batches of diverse graphs with different sizes, thanks to shared feature transforms and metrics.
• Experiments on Delaney, Az-logD, NIH-NCI, Hydration-free energy, Tox21, ClinTox, Sider, Toxcast, and Sydney point clouds demonstrate broad effectiveness of AGCN across regression, classification, and multi-task scenarios.
• Table 1 shows AGCN achieving lower RMSE than graphconv, NFP, and GCN across several molecular datasets.