[Paper Review] subgraph2vec: Learning Distributed Representations of Rooted Sub-graphs from Large Graphs

A. Sankara Narayanan, Mahinthan Chandramohan+3 more|arXiv (Cornell University)|Jun 29, 2016

Advanced Graph Neural Networks6 references139 citations

TL;DR

subgraph2vec learns unsupervised distributed representations of rooted subgraphs from large graphs and uses them with classifiers or deep models to improve graph-related tasks, outperforming traditional graph kernels.

ABSTRACT

In this paper, we present subgraph2vec, a novel approach for learning latent representations of rooted subgraphs from large graphs inspired by recent advancements in Deep Learning and Graph Kernels. These latent representations encode semantic substructure dependencies in a continuous vector space, which is easily exploited by statistical models for tasks such as graph classification, clustering, link prediction and community detection. subgraph2vec leverages on local information obtained from neighbourhoods of nodes to learn their latent representations in an unsupervised fashion. We demonstrate that subgraph vectors learnt by our approach could be used in conjunction with classifiers such as CNNs, SVMs and relational data clustering algorithms to achieve significantly superior accuracies. Also, we show that the subgraph vectors could be used for building a deep learning variant of Weisfeiler-Lehman graph kernel. Our experiments on several benchmark and large-scale real-world datasets reveal that subgraph2vec achieves significant improvements in accuracies over existing graph kernels on both supervised and unsupervised learning tasks. Specifically, on two realworld program analysis tasks, namely, code clone and malware detection, subgraph2vec outperforms state-of-the-art kernels by more than 17% and 4%, respectively.

Motivation & Objective

• Motivate the need for latent subgraph representations that capture semantic substructure dependencies in large graphs.
• Develop an unsupervised method to learn latent representations of rooted subgraphs using local neighborhood information.
• Demonstrate how subgraph vectors can enhance downstream tasks such as graph classification, clustering, link prediction, and community detection.
• Show that subgraph2vec can underpin a deep learning variant of the Weisfeiler-Lehman graph kernel.
• Validate the approach on benchmark and large-scale real-world datasets, including code clone and malware detection tasks.

Proposed method

• Leverage local neighborhood information around nodes to learn latent rooted-subgraph representations in an unsupervised manner.
• Represent rooted subgraphs as continuous vectors suitable for input to CNNs, SVMs, and relational clustering algorithms.
• Integrate the subgraph vectors into a deep learning variant of the Weisfeiler-Lehman graph kernel.
• Evaluate the learned representations on diverse tasks to demonstrate accuracy improvements over existing graph kernels.
• Show applicability to large-scale graphs and real-world datasets.

Experimental results

Research questions

• RQ1 Can latent representations of rooted subgraphs be learned from large graphs in an unsupervised fashion?
• RQ2 Do subgraph vectors improve performance on graph classification, clustering, link prediction, and community detection compared to existing graph kernels?
• RQ3 Can the subgraph representations be used to create a deep learning variant of the Weisfeiler-Lehman graph kernel?
• RQ4 Do experiments on benchmark and real-world datasets show significant accuracy gains with subgraph2vec?
• RQ5 Specifically, how do subgraph2vec-based methods perform on code clone and malware detection tasks?

Key findings

• Subgraph vectors can be used with classifiers such as CNNs and SVMs to achieve superior accuracies compared to traditional kernels.
• The approach enables a deep learning variant of the Weisfeiler-Lehman graph kernel.
• Experiments on benchmark and large-scale real-world datasets show significant accuracy improvements over existing graph kernels for supervised and unsupervised tasks.
• On code clone tasks, subgraph2vec outperforms state-of-the-art kernels by more than 17%.
• On malware detection tasks, subgraph2vec outperforms state-of-the-art kernels by more than 4%.