[Paper Review] Permutation invariant graph-to-sequence model for template-free retrosynthesis and reaction prediction

Zhengkai Tu, Connor W. Coley|arXiv (Cornell University)|Oct 19, 2021

Machine Learning in Materials Science61 references4 citations

TL;DR

This paper proposes Graph2SMILES, a permutation-invariant graph-to-sequence model that replaces SMILES-based input with molecular graph encoding to improve template-free retrosynthesis and reaction prediction. By combining a D-MPNN for local atomic interactions with a global attention encoder and graph-aware positional embeddings, it achieves state-of-the-art top-1 accuracy without requiring input-side SMILES augmentation, outperforming Transformers by 9.8% on USPTO_50k retrosynthesis.

ABSTRACT

Synthesis planning and reaction outcome prediction are two fundamental problems in computer-aided organic chemistry for which a variety of data-driven approaches have emerged. Natural language approaches that model each problem as a SMILES-to-SMILES translation lead to a simple end-to-end formulation, reduce the need for data preprocessing, and enable the use of well-optimized machine translation model architectures. However, SMILES representations are not an efficient representation for capturing information about molecular structures, as evidenced by the success of SMILES augmentation to boost empirical performance. Here, we describe a novel Graph2SMILES model that combines the power of Transformer models for text generation with the permutation invariance of molecular graph encoders that mitigates the need for input data augmentation. As an end-to-end architecture, Graph2SMILES can be used as a drop-in replacement for the Transformer in any task involving molecule(s)-to-molecule(s) transformations. In our encoder, an attention-augmented directed message passing neural network (D-MPNN) captures local chemical environments, and the global attention encoder allows for long-range and intermolecular interactions, enhanced by graph-aware positional embedding. Graph2SMILES improves the top-1 accuracy of the Transformer baselines by $1.7\%$ and $1.9\%$ for reaction outcome prediction on USPTO_480k and USPTO_STEREO datasets respectively, and by $9.8\%$ for one-step retrosynthesis on the USPTO_50k dataset.

Motivation & Objective

• Address the limitations of SMILES representations in molecular machine learning, which are sensitive to atom ordering and require data augmentation for robust performance.
• Develop a graph-to-sequence architecture that maintains permutation invariance to eliminate the need for input-side SMILES augmentation in retrosynthesis and reaction prediction tasks.
• Improve model performance on standard benchmarks—especially top-1 accuracy—by leveraging graph structure and attention mechanisms without relying on templates, atom mapping, or pretraining.
• Design a generalizable, plug-and-play backbone model that can replace the Transformer in any molecule-to-molecule transformation pipeline.

Proposed method

• Propose a two-stage encoder: an attention-augmented directed message passing neural network (D-MPNN) to capture local chemical environments, followed by a global self-attention encoder to model long-range and intermolecular interactions.
• Introduce graph-aware positional embeddings that encode topological information of molecular graphs, enhancing attention mechanisms without requiring pretraining or auxiliary tasks.
• Use a Transformer decoder to generate SMILES strings from the encoded molecular graph, treating both retrosynthesis and reaction prediction as graph-to-sequence tasks.
• Train the model end-to-end using cross-entropy loss for sequence generation, with no reliance on SMILES sequence input for the encoder.
• Ensure permutation invariance by encoding molecular structure directly via graph representation, making the model invariant to atom ordering in the input.
• Enable generalization to multi-molecule inputs (e.g., reactants and products) by extending the graph representation and attention mechanism to handle multiple subgraphs.

Experimental results

Research questions

• RQ1 Can a graph-to-sequence model achieve superior performance in template-free retrosynthesis and reaction prediction without relying on SMILES augmentation?
• RQ2 To what extent do graph-aware positional embeddings and global attention improve performance over standard Transformer encoders in molecular generation tasks?
• RQ3 Does a permutation-invariant graph encoder eliminate the need for input-side SMILES augmentation while maintaining or improving model accuracy?
• RQ4 How does the proposed Graph2SMILES model compare to SOTA methods that use templates, atom mapping, or pretraining on standard benchmarks?
• RQ5 What is the relative contribution of each component (D-MPNN, positional embedding, global attention) to the final performance of the model?

Key findings

• Graph2SMILES achieves a top-1 accuracy of 52.9% on the USPTO_50k retrosynthesis dataset, outperforming Transformer baselines by 9.8% without using templates, atom mapping, or input-side SMILES augmentation.
• On the USPTO_480k and USPTO_STEREO datasets, Graph2SMILES improves top-1 accuracy by 1.7% and 1.9%, respectively, over Transformer baselines in reaction outcome prediction.
• Ablation studies show that removing the global attention encoder reduces top-1 accuracy by 8.3 points, and removing graph-aware positional embeddings reduces it by 2.1 points, confirming the importance of both components.
• The model demonstrates strong generalization to multi-molecule inputs and maintains permutation invariance, eliminating the need for data augmentation based on equivalent SMILES strings.
• Graph2SMILES is a drop-in replacement for the Transformer in molecular transformation tasks, enabling performance gains without architectural overhaul or dependency on auxiliary training.
• The model’s performance is robust and competitive even without output-side SMILES augmentation, though future work may explore variational inference to improve top-n generation quality.