[Paper Review] Generative Artificial Intelligence for Navigating Synthesizable Chemical Space

Wenhao Gao, Shitong Luo+1 more|arXiv (Cornell University)|Oct 4, 2024

Advanced Data Processing Techniques13 citations

TL;DR

SynFormer is a transformer-based framework that generates synthesizable synthetic pathways, enabling local and global exploration of synthesizable chemical space with diffusion-based building block selection and end-to-end differentiability.

ABSTRACT

We introduce SynFormer, a generative modeling framework designed to efficiently explore and navigate synthesizable chemical space. Unlike traditional molecular generation approaches, we generate synthetic pathways for molecules to ensure that designs are synthetically tractable. By incorporating a scalable transformer architecture and a diffusion module for building block selection, SynFormer surpasses existing models in synthesizable molecular design. We demonstrate SynFormer's effectiveness in two key applications: (1) local chemical space exploration, where the model generates synthesizable analogs of a reference molecule, and (2) global chemical space exploration, where the model aims to identify optimal molecules according to a black-box property prediction oracle. Additionally, we demonstrate the scalability of our approach via the improvement in performance as more computational resources become available. With our code and trained models openly available, we hope that SynFormer will find use across applications in drug discovery and materials science.

Motivation & Objective

• Motivate the need for synthesis-centric molecular design that guarantees synthetic tractability.
• Develop a scalable generative framework that produces synthetic pathways rather than just structures.
• Leverage a transformer backbone with a diffusion module to select building blocks and reactions.
• Demonstrate local (reference-molecule–driven) and global (black-box objective–driven) space exploration of synthesizable chemistry.

Proposed method

• Represent synthetic pathways in a postfix notation with tokens for START, END, RXN, and BB.
• Use a transformer to autoregressively generate pathway tokens and classify token types at each step.
• Incorporate a denoising diffusion probabilistic module to predict building-block fingerprints and select BBs.
• Train two instantiations: SynFormer-D (decoder-only) and SynFormer-ED (encoder–decoder conditioned on input SMILES).
• Train on a simulated space built from 115 reaction templates and 223,244 building blocks, extending Enamine REAL Space.

Figure 1: Schematic illustration of the SynFormer framework and architecture. (A) The SynFormer-ED architecture is an encoder-decoder that takes a molecule as input and outputs a synthetic route to the same or an analogous molecule. (B) SynFormer-D is a decoder-only framework designed to generate sy

Experimental results

Research questions

• RQ1 Can SynFormer accurately reconstruct molecules and cover a large, synthesizable chemical space?
• RQ2 Can SynFormer generate synthesizable analogs for unsynthesizable inputs while preserving key features?
• RQ3 Can SynFormer navigate global chemical space to optimize properties while respecting synthetic feasibility?
• RQ4 How does SynFormer perform as a mutation operator or within reinforcement-learning–guided optimization frameworks?

Key findings

• SynFormer-ED achieves higher reconstruction rates on REAL Space (66%) than previous models and improves over ChEMBL (20%).
• Model performance (fingerprint BCE) improves with increased model size and data; performance scaling requires both more data and compute.
• SynFormer-ED can generate synthesizable analogs for unsynthesizable designs and preserve objective scores while improving synthetic accessibility.
• SynFormer-D fine-tuned with RL (SF-RL) can bias generation toward high-scoring molecules for DRD2 binding and outperforms several methods in certain setups.
• Using SynFormer-ED as a mutation operator within GraphGA (GraphGA-SF) yields competitive optimization with enhanced synthetic feasibility across GuacaMol tasks.
• The framework enables local space projection, hit expansion, and global optimization while ensuring synthetic routes exist.

Figure 2: Model performance on molecular reconstruction. (A and B) Comparison of the reconstruction rate and average structural (Tanimoto) similarity between input and output molecules for SynFormer-ED, ChemProjector [ 67 ] , and SynNet [ 65 ] on 1,000 randomly selected molecules from (A) REAL Diver