[Paper Review] Wasserstein Weisfeiler-Lehman Graph Kernels

Matteo Togninalli, Elisabetta Ghisu+3 more|arXiv (Cornell University)|Jun 4, 2019

Graph Theory and Algorithms43 references62 citations

TL;DR

WWL combines a Weisfeiler–Lehman embedding with the Wasserstein distance to compare graphs by distributions of node features, achieving strong performance on continuous attributes and competitive results on categorical labels.

ABSTRACT

Most graph kernels are an instance of the class of $\mathcal{R}$-Convolution kernels, which measure the similarity of objects by comparing their substructures. Despite their empirical success, most graph kernels use a naive aggregation of the final set of substructures, usually a sum or average, thereby potentially discarding valuable information about the distribution of individual components. Furthermore, only a limited instance of these approaches can be extended to continuously attributed graphs. We propose a novel method that relies on the Wasserstein distance between the node feature vector distributions of two graphs, which allows to find subtler differences in data sets by considering graphs as high-dimensional objects, rather than simple means. We further propose a Weisfeiler-Lehman inspired embedding scheme for graphs with continuous node attributes and weighted edges, enhance it with the computed Wasserstein distance, and thus improve the state-of-the-art prediction performance on several graph classification tasks.

Motivation & Objective

• Motivate graph kernels that better capture substructure distributions rather than simple sums/averages.
• Introduce a graph Wasserstein distance (GWD) between node feature sets.
• Develop a WL-inspired embedding scheme that handles continuous attributes and weighted edges.
• Combine GWD with a Laplacian-like WWL kernel for improved graph classification.
• Evaluate WWL on benchmarks with both categorical labels and continuous attributes.

Proposed method

• Define graph embedding scheme f that outputs node embeddings for each graph.
• Define Graph Wasserstein Distance D^f_W(G,G') as W1(f(G),f(G')).
• Extend WL to continuous attributes via an averaging propagation step accounting for edge weights.
• Construct WWL kernel K_WWL = exp(-λ D^f_W) as a Laplacian-like kernel.
• Differentiate categorical WWL (positive definite) from continuous WWL (indefinite in theory, tackled with Kreĭn-SVM).
• Optionally use Sinkhorn regularisation to speed up Wasserstein computations for large graphs.

Experimental results

Research questions

• RQ1 Can Wasserstein distance between node-embedding distributions capture finer graph similarities than traditional substructure aggregations?
• RQ2 How can WL-like embeddings be extended to continuously attributed graphs with weighted edges?
• RQ3 Does WWL improve graph classification on benchmarks with continuous node attributes while remaining competitive on categorical-label datasets?

Key findings

• WWL matches state-of-the-art on categorically labelled graphs (comparable to WL-OA).
• WWL significantly outperforms baselines on graphs with continuous node attributes across several datasets.
• The categorical WWL kernel is proven positive definite for all λ>0.
• WWL achieves top average ranking in continuous-attribute experiments across multiple datasets.
• RBF-WL baselines and other graph kernels are outperformed by WWL on several continuous-attribute tasks.
• WWL benefits from using optimal transport to compare whole distributions of node features rather than mere aggregates.