[Paper Review] Linking the Neural Machine Translation and the Prediction of Organic Chemistry Reactions

Juno Nam, Jurae Kim|arXiv (Cornell University)|Dec 29, 2016

Topic Modeling6 references122 citations

TL;DR

This paper casts reaction product prediction as neural machine translation, using a gated recurrent unit sequence-to-sequence model with a SMILES tokenizer and attention to translate reactants and reagents into products, trained on patent reactions and Wade textbook reactions.

ABSTRACT

Finding the main product of a chemical reaction is one of the important problems of organic chemistry. This paper describes a method of applying a neural machine translation model to the prediction of organic chemical reactions. In order to translate 'reactants and reagents' to 'products', a gated recurrent unit based sequence-to-sequence model and a parser to generate input tokens for model from reaction SMILES strings were built. Training sets are composed of reactions from the patent databases, and reactions manually generated applying the elementary reactions in an organic chemistry textbook of Wade. The trained models were tested by examples and problems in the textbook. The prediction process does not need manual encoding of rules (e.g., SMARTS transformations) to predict products, hence it only needs sufficient training reaction sets to learn new types of reactions.

Motivation & Objective

• Motivate predicting main reaction products without manually encoded transformation rules.
• Represent reactions as SMILES and treat prediction as translation from reactants/reagents to products.
• Evaluate how real patent-derived training data vs. generated template-based data affect prediction performance.

Proposed method

• Tokenize SMILES representations of reactions using a PEG-based parser to create input/output token sequences.
• Use a 3-layer GRU encoder–decoder with attention to model p(y|x) for product generation from reversed input tokens.
• Train two models: one on real patent-based reactions and one on generated Wade-template reactions, then compare them.
• Normalize and bucket input/output sequences for scalable training with 600-dim embeddings.
• Process training data by removing atom mappings and filtering extreme reaction examples to fit model input constraints.

Experimental results

Research questions

• RQ1 Can a neural translation model predict organic reaction products without hand-crafted SMARTS rules?
• RQ2 Does training on real patent reactions improve prediction over training on generated elementary-reaction templates?
• RQ3 How well does the model generalize to longer input sequences and more complex substrates?
• RQ4 What is the impact of attention mechanisms on aligning input tokens to product generation?

Key findings

• A real-patent–based training set improves product prediction over using only generated template reactions.
• The model can extrapolate to some unencoded reaction patterns, including aromatic substrates.
• Longer input sequences (more atoms) increase error rates and reduce fully correct predictions, but the real+gen model maintains relatively stable Tanimoto similarity and lower invalid-SMILES rates.
• The attention mechanism aligns decoder steps to encoder tokens, suggesting potential improvements by better mapping reactive sites.
• The model can generate valid product SMILES more reliably when trained on larger, more diverse datasets (real+gen) than on generated data alone.