[Paper Review] SMILES-X: autonomous molecular compounds characterization for small datasets without descriptors

G. Lambard, Ekaterina Gracheva|arXiv (Cornell University)|Jun 20, 2019

Machine Learning in Materials ScienceMaterials Science65 references26 citations

TL;DR

SMILES-X is a novel neural architecture for autonomous molecular property prediction on small datasets using only SMILES strings as input. It employs SMILES augmentation, a custom {Embed-Encode-Attend-Predict} transformer-like architecture with attention-based interpretability, and Bayesian hyperparameter optimization to achieve state-of-the-art performance in predicting aqueous solubility (RMSE ≈ 0.57 mols/L), hydration free energy (RMSE ≈ 0.81 kcal/mol), and LogD (RMSE ≈ 0.59), without relying on handcrafted descriptors or large datasets.

ABSTRACT

There is more and more evidence that machine learning can be successfully applied in materials science and related fields. However, datasets in these fields are often quite small ($\ll1000$ samples). It makes the most advanced machine learning techniques remain neglected, as they are considered to be applicable to big data only. Moreover, materials informatics methods often rely on human-engineered descriptors, that should be carefully chosen, or even created, to fit the physicochemical property that one intends to predict. In this article, we propose a new method that tackles both the issue of small datasets and the difficulty of task-specific descriptors development. The SMILES-X is an autonomous pipeline for molecular compounds characterisation based on a \{Embed-Encode-Attend-Predict\} neural architecture with a data-specific Bayesian hyper-parameters optimisation. The only input to the architecture -- the SMILES strings -- are de-canonicalised in order to efficiently augment the data. One of the key features of the architecture is the attention mechanism, which enables the interpretation of output predictions without extra computational cost. The SMILES-X shows new state-of-the-art results in the inference of aqueous solubility ($\overline{RMSE}_{test} \simeq 0.57 \pm 0.07$ mols/L), hydration free energy ($\overline{RMSE}_{test} \simeq 0.81 \pm 0.22$ kcal/mol, which is $\sim 24.5\%$ better than molecular dynamics simulations), and octanol/water distribution coefficient ($\overline{RMSE}_{test} \simeq 0.59 \pm 0.02$ for LogD at pH 7.4) of molecular compounds. The SMILES-X is intended to become an important asset in the toolkit of materials scientists and chemists. The source code for the SMILES-X is available at \href{https://github.com/GLambard/SMILES-X}{github.com/GLambard/SMILES-X}.

Motivation & Objective

• Address the challenge of training deep learning models on small molecular datasets (<1000 samples), which are common in materials science.
• Eliminate the need for human-engineered molecular descriptors by using raw SMILES strings as input.
• Enable interpretable predictions through an attention mechanism that highlights chemically relevant fragments without additional training.
• Develop an end-to-end, autonomous pipeline that optimizes model hyperparameters per dataset using Bayesian optimization.
• Achieve state-of-the-art performance in predicting key physicochemical properties such as solubility, hydration free energy, and lipophilicity.

Proposed method

• Uses de-canonicalized SMILES strings as the sole input representation to increase data diversity through SMILES augmentation.
• Applies a data-specific Bayesian optimization to tune neural network hyperparameters, improving generalization on small datasets.
• Employs a {Embed-Encode-Attend-Predict} neural architecture with an attention mechanism to focus on relevant molecular fragments during prediction.
• Leverages attention weights to enable real-time, model-agnostic interpretation of predictions by highlighting influential SMILES tokens.
• Uses temporal relative distance (Tdist) to track how prediction evolves as the SMILES is read token by token, enabling dynamic insight into structural influence.
• Reduces model complexity and overfitting risk by minimizing trainable parameters through attention-based feature selection.

Experimental results

Research questions

• RQ1 Can a deep learning model achieve state-of-the-art performance in predicting molecular physicochemical properties without using handcrafted descriptors?
• RQ2 How effective is SMILES augmentation (via de-canonicalization and reindexing) in improving model generalization on small datasets?
• RQ3 To what extent can attention mechanisms in a lightweight neural architecture provide interpretable predictions without additional training?
• RQ4 Can Bayesian hyperparameter optimization significantly improve performance on small molecular datasets compared to random or grid search?
• RQ5 Does the SMILES-X pipeline outperform existing methods in predicting solubility, hydration free energy, and LogD on benchmark datasets like MoleculeNet?

Key findings

• SMILES-X achieves a test RMSE of 0.57 ± 0.07 mols/L for aqueous solubility prediction on the ESOL dataset, setting a new state-of-the-art.
• For hydration free energy prediction on the FreeSolv dataset, SMILES-X achieves an RMSE of 0.81 ± 0.22 kcal/mol, representing a 24.5% improvement over molecular dynamics simulations.
• On the Lipophilicity dataset, SMILES-X achieves an RMSE of 0.59 ± 0.02 for LogD at pH 7.4, demonstrating high accuracy on lipophilicity prediction.
• The attention mechanism successfully identifies chemically relevant fragments—such as the oxygen atom in a phenolic compound—as the most influential for hydration free energy prediction.
• The temporal relative distance (Tdist) analysis shows that predictions stabilize quickly, with significant convergence observed after processing only the first few tokens of the SMILES string.
• SMILES augmentation without canonicalization leads to a measurable performance gain (Can vs. Augm), which would not be feasible with graph-based architectures.