[Paper Review] Application of generative autoencoder in de novo molecular design

Thomas Blaschke, Marcus Olivecrona+3 more|arXiv (Cornell University)|Nov 21, 2017

Computational Drug Discovery Methods25 references76 citations

TL;DR

The paper evaluates several generative autoencoder architectures (VAE and AAE variants) for de novo molecular design, showing latent-space structure preservation and enabling Bayesian optimization to target predicted DRD2 activity.

ABSTRACT

A major challenge in computational chemistry is the generation of novel molecular structures with desirable pharmacological and physiochemical properties. In this work, we investigate the potential use of autoencoder, a deep learning methodology, for de novo molecular design. Various generative autoencoders were used to map molecule structures into a continuous latent space and vice versa and their performance as structure generator was assessed. Our results show that the latent space preserves chemical similarity principle and thus can be used for the generation of analogue structures. Furthermore, the latent space created by autoencoders were searched systematically to generate novel compounds with predicted activity against dopamine receptor type 2 and compounds similar to known active compounds not included in the training set were identified.

Motivation & Objective

• Explore whether compounds can be mapped into a continuous latent space and reconstructed by autoencoders.
• Assess if the latent space preserves chemical similarity principles.
• Evaluate if latent-space representations can be used to generate novel compounds with desired properties.
• Compare different autoencoder architectures (VAE with/without teacher forcing, AAE with Gaussian/Uniform priors).
• Demonstrate Bayesian optimization in latent space guided by a QSAR model for target activity (DRD2).

Proposed method

• Train multiple autoencoder models (NoTeacher VAE, Teacher VAE, Gauss AAE, Uniform AAE) on ChEMBL SMILES data and map to a 56-dimensional latent space.
• Use teachers forcing to improve SMILES validity during sequence generation.
• Implement adversarial autoencoders by adding a discriminator to enforce latent priors (Gaussian or Uniform).
• Tokenize SMILES into 35 tokens and generate new SMILES by sampling decoder outputs and validating with RDKit.
• Apply Bayesian optimization in latent space with a DRD2 activity SVM model as the objective to identify high-scoring compounds.
• Filter generated structures to remove rings larger than eight atoms and enforce SMILES validity.

Experimental results

Research questions

• RQ1 Can compounds be accurately mapped to and reconstructed from a continuous latent space in generative autoencoders?
• RQ2 Does the latent space preserve local chemical similarity to enable analogue searching?
• RQ3 Can latent-space generation coupled with a QSAR model discover novel compounds with predicted activity against DRD2?
• RQ4 How do different AE architectures affect reconstruction, validity of generated SMILES, and latent-space smoothness?
• RQ5 Is Bayesian optimization effective at steering latent-space searches toward high-activity candidates while maintaining synthetic feasibility?

Key findings

• All four models achieve high character reconstruction on training data, with teacher-forced models yielding higher percentages of valid SMILES in generation mode.
• Uniform AAE provides the largest fraction of valid SMILES and maintains a smooth latent space even when Celecoxib analogues are removed from training.
• Latent-space proximity to Celecoxib correlates with higher structural similarity in generated compounds, supporting the similarity principle locally.
• BO-guided search in the Uniform AAE latent space yields structures with high predicted DRD2 activity and reasonable similarity to known actives.
• The BO approach identifies many candidates with Pactive values above 0.5, and a substantial portion of generated actives are within reasonable similarity to validated actives.
• When Celecoxib analogues are excluded from training, the Uniform AAE still reconstructs Celecoxib neighbourhoods effectively, demonstrating robustness of the latent representation.