[Paper Review] SMILES Enumeration as Data Augmentation for Neural Network Modeling of Molecules

Esben Jannik Bjerrum|arXiv (Cornell University)|Mar 21, 2017

Computational Drug Discovery MethodsComputer Science6 references301 citations

TL;DR

The paper demonstrates data augmentation by enumerating non-canonical SMILES to train an LSTM-based QSAR model, improving predictive performance and enabling better averaging of predictions across enumerated SMILES per molecule.

ABSTRACT

Simplified Molecular Input Line Entry System (SMILES) is a single line text representation of a unique molecule. One molecule can however have multiple SMILES strings, which is a reason that canonical SMILES have been defined, which ensures a one to one correspondence between SMILES string and molecule. Here the fact that multiple SMILES represent the same molecule is explored as a technique for data augmentation of a molecular QSAR dataset modeled by a long short term memory (LSTM) cell based neural network. The augmented dataset was 130 times bigger than the original. The network trained with the augmented dataset shows better performance on a test set when compared to a model built with only one canonical SMILES string per molecule. The correlation coefficient R2 on the test set was improved from 0.56 to 0.66 when using SMILES enumeration, and the root mean square error (RMS) likewise fell from 0.62 to 0.55. The technique also works in the prediction phase. By taking the average per molecule of the predictions for the enumerated SMILES a further improvement to a correlation coefficient of 0.68 and a RMS of 0.52 was found.

Motivation & Objective

• Motivate data augmentation for small molecular QSAR datasets using SMILES diversity.
• Show that enumerating SMILES improves neural network training stability and generalization.
• Evaluate predictive performance with canonical vs. enumerated SMILES in train/test settings.
• Demonstrate the benefit of averaging predictions across enumerated SMILES for the same molecule.

Proposed method

• Generate multiple SMILES per molecule by randomizing atom ordering and using RDKit with canonical SMILES disabled.
• Construct a fixed-length one-hot encoded representation of SMILES strings.
• Train LSTM-based QSAR models with and without SMILES enumeration using Keras/Theano back end.
• Hyperparameter optimization via Bayesian optimization (Gaussian process) over a defined space.
• Compare performance on canonical-only vs. enumerated datasets in train/test settings.
• Compute and compare metrics such as R^2 and RMS for predictions; also evaluate averaged enumerated predictions per molecule.

Experimental results

Research questions

• RQ1 Does SMILES enumeration as data augmentation improve QSAR model performance compared to using canonical SMILES alone?
• RQ2 How does training with enumerated SMILES affect generalization to non-canonical SMILES in both training and test sets?
• RQ3 What is the effect of averaging predictions across enumerated SMILES on predictive accuracy?
• RQ4 What are the optimal hyperparameters when using canonical versus enumerated SMILES for LSTM-QSAR models?
• RQ5 Is the augmentation beneficial for both training and prediction phases?

Key findings

• Enumerated SMILES dataset augmented the training data by roughly 130x (602 train canonical vs. 79,143 train enumerated; test 71 canonical vs. 9,412 enumerated).
• Best canonical-model test R^2 = 0.56 and RMS = 0.62; best enumerated-model test R^2 = 0.66 and RMS = 0.55.
• Prediction of test set using averaged enumerated SMILES per molecule yielded R^2 = 0.68 and RMS = 0.52.
• Enumerated-model trained on enumerated data achieved higher training R^2 (0.87) and lower training RMS (0.37) than canonical-model (R^2 0.85, RMS 0.39).
• The worst case occurred when using the enumerated SMILES with the canonical-trained model (R^2 = 0.26, RMS = 0.84).
• Training with enumerated data produced less noisy convergence and faster effective updates due to larger sample size.