[Paper Review] G-MATT: Single-step Retrosynthesis Prediction using Molecular Grammar Tree Transformer

Guomin Zhang, Vipul Mann+1 more|arXiv (Cornell University)|May 4, 2023

Machine Learning in Materials ScienceMaterials Science3 citations

TL;DR

G-MATT proposes a novel tree-to-sequence transformer model that uses hierarchical SMILES grammar trees to encode molecular structure with explicit chemistry-aware representations, significantly improving single-step retrosynthesis prediction. It achieves 51% top-1 accuracy, 79.1% top-10 accuracy, 1.5% invalid rate, and 74.8% bioactive similarity on the USPTO-50K dataset, outperforming standard SMILES-based models by integrating structural and functional group information.

ABSTRACT

Various template-based and template-free approaches have been proposed for single-step retrosynthesis prediction in recent years. While these approaches demonstrate strong performance from a data-driven metrics standpoint, many model architectures do not incorporate underlying chemistry principles. Here, we propose a novel chemistry-aware retrosynthesis prediction framework that combines powerful data-driven models with prior domain knowledge. We present a tree-to-sequence transformer architecture that utilizes hierarchical SMILES grammar-based trees, incorporating crucial chemistry information that is often overlooked by SMILES text-based representations, such as local structures and functional groups. The proposed framework, grammar-based molecular attention tree transformer (G-MATT), achieves significant performance improvements compared to baseline retrosynthesis models. G-MATT achieves a promising top-1 accuracy of 51% (top-10 accuracy of 79.1%), invalid rate of 1.5%, and bioactive similarity rate of 74.8% on the USPTO- 50K dataset. Additional analyses of G-MATT attention maps demonstrate the ability to retain chemistry knowledge without relying on excessively complex model architectures.

Motivation & Objective

• To address the limitation of SMILES text-based representations in capturing essential chemical structure and functional group information for retrosynthesis prediction.
• To integrate domain-specific chemical grammar rules into deep learning models to improve interpretability and performance.
• To develop a hierarchical, tree-structured molecular representation that preserves local bonding and functional group context.
• To achieve state-of-the-art performance in single-step retrosynthesis prediction while maintaining chemical plausibility and low invalid reaction rates.
• To demonstrate that attention mechanisms in the model can retain chemistry knowledge without requiring overly complex architectures.

Proposed method

• The model uses a tree-to-sequence transformer architecture where molecules are represented as SMILES grammar trees, capturing hierarchical structural relationships.
• Each molecule is parsed into a tree using a custom SMILES grammar, encoding atoms, bonds, branching, rings, and stereochemistry explicitly.
• The encoder processes the grammar tree representation of the target molecule, while the decoder attends to reactant SMILES strings via cross-attention.
• The model leverages self-attention and cross-attention mechanisms to align structural motifs between the target and predicted precursors.
• A custom set of 24 grammar rules defines the hierarchical structure of SMILES, enabling precise parsing into atomic, bond, and branching components.
• The architecture is trained end-to-end on the USPTO-50K dataset using standard sequence-to-sequence objectives with cross-entropy loss.

Experimental results

Research questions

• RQ1 Can a grammar-based molecular tree representation improve the accuracy and chemical plausibility of retrosynthesis predictions compared to standard SMILES string modeling?
• RQ2 To what extent can attention mechanisms in a transformer model learn chemically meaningful alignments between target molecules and their precursors when using hierarchical tree structures?
• RQ3 Does integrating explicit chemical grammar rules into the model architecture reduce the invalid reaction rate while maintaining high prediction accuracy?
• RQ4 How does the model’s performance compare to existing template-free and template-based retrosynthesis models on standard benchmarks?
• RQ5 Can attention visualization confirm that the model focuses on chemically relevant functional groups and reaction centers during prediction?

Key findings

• G-MATT achieves a top-1 accuracy of 51% on the USPTO-50K dataset, significantly outperforming baseline SMILES-based models.
• The model attains a top-10 accuracy of 79.1%, indicating strong coverage of plausible retrosynthetic pathways.
• The invalid reaction rate is only 1.5%, demonstrating high syntactic and chemical correctness in generated precursors.
• The bioactive similarity rate reaches 74.8%, indicating that predicted precursors are structurally similar to bioactive compounds in the training set.
• Attention visualization confirms that the model attends to chemically relevant fragments such as reaction centers, functional groups, and stereochemistry.
• The model maintains high performance without requiring complex architectural modifications, suggesting that grammar-based tree representation alone enhances learning efficiency.