[Paper Review] A Deep Generative Model for Fragment-Based Molecule Generation

Marco Podda, Davide Bacciu+1 more|arXiv (Cornell University)|Feb 28, 2020

Machine Learning in Materials Science29 references35 citations

TL;DR

The paper introduces a fragment-based language model for molecule generation that uses BRICS-based fragmentation and low-frequency masking to improve validity and uniqueness, achieving competitive results with graph-based methods.

ABSTRACT

Molecule generation is a challenging open problem in cheminformatics. Currently, deep generative approaches addressing the challenge belong to two broad categories, differing in how molecules are represented. One approach encodes molecular graphs as strings of text, and learns their corresponding character-based language model. Another, more expressive, approach operates directly on the molecular graph. In this work, we address two limitations of the former: generation of invalid and duplicate molecules. To improve validity rates, we develop a language model for small molecular substructures called fragments, loosely inspired by the well-known paradigm of Fragment-Based Drug Design. In other words, we generate molecules fragment by fragment, instead of atom by atom. To improve uniqueness rates, we present a frequency-based masking strategy that helps generate molecules with infrequent fragments. We show experimentally that our model largely outperforms other language model-based competitors, reaching state-of-the-art performances typical of graph-based approaches. Moreover, generated molecules display molecular properties similar to those in the training sample, even in absence of explicit task-specific supervision.

Motivation & Objective

• Address high invalidity and duplicates in SMILES-based generators by leveraging fragment-level generation inspired by Fragment-Based Drug Design (FBDD).
• Increase molecule validity by generating chemically sound fragments and valid connections between fragments.
• Enhance molecule uniqueness by a masking strategy that boosts rare fragments during generation.
• Demonstrate competitive performance against graph-based generators despite using a simpler, fragment-based representation.

Proposed method

• Fragment molecules into an ordered sequence of BRICS fragments using a left-to-right SMILES-guided fragmentation with dummy attachment points.
• Construct a fragment vocabulary and train fragment embeddings via a skip-gram model with negative sampling to capture contextual similarity.
• Use an encoder-decoder VAE-like architecture with GRU-based encoder and decoder; latent z is sampled and used to initialize the decoder.
• Train with teacher forcing to maximize the log-likelihood of fragment sequences and regularize with a KL divergence term to Gaussian space.
• Introduce Low-Frequency Masking to replace rare fragments with frequency-based tokens, enabling sampling of infrequent fragments during generation.
• During generation, sample latent z, decode fragment sequences with greedy sampling, and reconstruct valid molecules from fragments; enforce attachment-point constraints.

Experimental results

Research questions

• RQ1 Can fragment-based generation improve validity rates compared to atom-by-atom SMILES-based models?
• RQ2 Does Low-Frequency Masking (LFM) improve uniqueness and diversity of generated molecules?
• RQ3 How does fragment-based LM-based generation compare to graph-based molecular generators in terms of validity, novelty, and uniqueness?

Key findings

• The fragment-based LM achieves perfect validity on ZINC and PCBA datasets, outperforming LM baselines and matching state-of-the-art graph-based methods in validity.
• Uniqueness improves for both LM and LFM variants, with larger gains when using LFM.
• LFM yields notable improvements in PCBA, bringing LM performance closer to state-of-the-art graph models.
• Compared to graph-based models, the LFM variant becomes competitive and often superior in uniqueness on ZINC/PCBA tasks.
• Generated molecules display structural features and drug-like properties similar to the training data even without task-specific supervision.