[Paper Review] N-Gram Graph: Simple Unsupervised Representation for Graphs, with Applications to Molecules

Shengchao Liu, Mehmet F. Demirel+1 more|arXiv (Cornell University)|Jun 24, 2018

Computational Drug Discovery MethodsComputer Science58 references78 citations

TL;DR

Introduces the N-gram Graph, an unsupervised, training-free graph representation for molecules that builds graph embeddings from vertex embeddings via short walks, achieving strong predictive performance across multiple molecule property tasks.

ABSTRACT

Machine learning techniques have recently been adopted in various applications in medicine, biology, chemistry, and material engineering. An important task is to predict the properties of molecules, which serves as the main subroutine in many downstream applications such as virtual screening and drug design. Despite the increasing interest, the key challenge is to construct proper representations of molecules for learning algorithms. This paper introduces the N-gram graph, a simple unsupervised representation for molecules. The method first embeds the vertices in the molecule graph. It then constructs a compact representation for the graph by assembling the vertex embeddings in short walks in the graph, which we show is equivalent to a simple graph neural network that needs no training. The representations can thus be efficiently computed and then used with supervised learning methods for prediction. Experiments on 60 tasks from 10 benchmark datasets demonstrate its advantages over both popular graph neural networks and traditional representation methods. This is complemented by theoretical analysis showing its strong representation and prediction power.

Motivation & Objective

• Motivate molecule property prediction and the need for effective graph representations.
• Propose an unsupervised, training-free graph representation based on vertex embeddings and n-gram walks.
• Show that the method is equivalent to a simple GNN with no parameters and training.
• Demonstrate strong predictive performance across 60 tasks from 10 benchmark datasets.
• Provide theoretical analysis of representation power and prediction guarantees.

Proposed method

• Embed graph vertices using attribute-aware one-hot encoding and learn a vertex embedding matrix W by predicting a vertex's attributes from neighbor attributes (unsupervised training).
• Construct n-gram embeddings by taking element-wise products of vertex embeddings along walks and summing over all walks of length n.
• Concatenate f_(1), ..., f_(T) to form the graph embedding f_G, representing the graph up to walk length T.
• Show the n-gram graph embedding is equivalent to a parameter-free Graph Neural Network with iterative neighbor aggregation.
• Provide runtime analysis: O(r T (m + m_e)) where r is embedding dim, T is walk length, m and m_e are vertices and edges.

Experimental results

Research questions

• RQ1 Can an unsupervised, simple graph representation achieve competitive molecule property predictions without training end-to-end on labeled data?
• RQ2 How much information about the graph is preserved in n-gram walk statistics when embedded via vertex embeddings?
• RQ3 Is the proposed N-gram graph representation transferable across datasets and tasks?
• RQ4 How do hyperparameters r (embedding dimension) and T (walk length) affect performance?

Key findings

• N-gram graph with simple XGBoost or Random Forest often outperforms traditional fingerprints and several GNNs across 60 tasks.
• Embeddings trained on one dataset can transfer to others and even random embeddings yield competitive results for some tasks.
• The method achieves superior or comparable performance on QM9/QM8 datasets compared to 3D-information-based models like DTNN/MPNN.
• Increasing T generally improves performance, while larger r has less impact.
• The approach is faster to construct representations than many GNNs and competitive with kernel-based methods.