[Paper Review] C5T5: Controllable Generation of Organic Molecules with Transformers

Daniel Rothchild, Alex Tamkin+3 more|arXiv (Cornell University)|Aug 23, 2021

Molecular Junctions and NanostructuresEngineering35 references17 citations

TL;DR

C5T5 proposes a self-supervised, transformer-based method for zero-shot, controllable generation of organic molecules by treating IUPAC names as natural language, enabling domain experts to make chemically intuitive select-and-replace edits to optimize molecular properties without paired edit data. It achieves interpretable, property-directed molecular modifications by conditioning on discretized property values and masking target fragments for infilling, demonstrating success in optimizing four drug-relevant physical properties with chemically valid and intuitive strategies.

ABSTRACT

Methods for designing organic materials with desired properties have high potential impact across fields such as medicine, renewable energy, petrochemical engineering, and agriculture. However, using generative modeling to design substances with desired properties is difficult because candidate compounds must satisfy multiple constraints, including synthetic accessibility and other metrics that are intuitive to domain experts but challenging to quantify. We propose C5T5, a novel self-supervised pretraining method that enables transformers to make zero-shot select-and-replace edits, altering organic substances towards desired property values. C5T5 operates on IUPAC names -- a standardized molecular representation that intuitively encodes rich structural information for organic chemists but that has been largely ignored by the ML community. Our technique requires no edited molecule pairs to train and only a rough estimate of molecular properties, and it has the potential to model long-range dependencies and symmetric molecular structures more easily than graph-based methods. C5T5 also provides a powerful interface to domain experts: it grants users fine-grained control over the generative process by selecting and replacing IUPAC name fragments, which enables experts to leverage their intuitions about structure-activity relationships. We demonstrate C5T5's effectiveness on four physical properties relevant for drug discovery, showing that it learns successful and chemically intuitive strategies for altering molecules towards desired property values.

Motivation & Objective

• To address the challenge of designing organic molecules with desired physical properties in drug discovery, where traditional methods are limited by vast chemical space and expert intuition is underutilized.
• To overcome the limitations of existing generative models that rely on atom-based representations like SMILES or molecular graphs, which lack intuitive structure for chemists.
• To develop a method that enables fine-grained, user-directed molecular optimization without requiring paired edit data or explicit demonstrations of desired transformations.
• To provide a human-in-the-loop interface that aligns with chemists' mental models of structure-activity relationships through IUPAC name manipulation.
• To enable zero-shot property-directed editing by training on self-supervised infilling with coarse property conditioning, allowing inference with desired target property values.

Proposed method

• C5T5 represents organic molecules using IUPAC names, a semantically rich, standardized representation that encodes functional groups and molecular symmetry in a way intuitive to chemists.
• The model uses a conditional T5-based transformer architecture trained on a self-supervised infilling objective, where masked tokens in the IUPAC name are predicted conditioned on the molecule’s discretized property value.
• During training, the model is conditioned on a discretized bucket of the target molecular property (e.g., logP, solubility, etc.) prepended to the IUPAC name.
• During inference, users select a fragment to replace, mask it with a sentinel token, and condition the model on a desired property bucket (e.g., <high> for increased logP), prompting the model to generate a new fragment.
• The method supports zero-shot editing: no paired edit examples are needed, as the model generalizes from self-supervised pretraining on unpaired molecules and property estimates.
• The interface allows domain experts to iteratively refine molecules by replacing fragments based on their chemical intuition, with the model generating chemically plausible and property-directed alternatives.

Experimental results

Research questions

• RQ1 Can a self-supervised language modeling approach on IUPAC names enable zero-shot, controllable molecular editing without requiring paired edit data or demonstrations?
• RQ2 How well can a transformer model learn to generate chemically intuitive and valid molecular modifications when conditioned only on coarse property estimates and masked fragments?
• RQ3 To what extent does using IUPAC names as a representation improve interpretability and alignment with expert intuition compared to SMILES or graph-based representations?
• RQ4 Can the model discover and apply known structure-activity relationships (e.g., adding hydrophobic chains to increase logP) without explicit supervision on such strategies?
• RQ5 How effective is the method in generating molecules with desired changes in multiple physical properties relevant to drug discovery, such as logP, solubility, and melting point?

Key findings

• C5T5 successfully learns to generate chemically intuitive edits that increase or decrease a molecule’s octanol-water partition coefficient (logP), such as replacing hydrophilic groups with long aliphatic chains to enhance hydrophobicity.
• The model generates symmetric molecules when the input IUPAC name encodes symmetry (e.g., 'bis' or 'diol' groups), demonstrating an ability to preserve or extend structural patterns.
• For molecules with symmetric scaffolds, C5T5 produces valid and symmetric outputs (e.g., replacing 'ol' with 'sulfinic acid' in a symmetric diol), showing robustness to structural complexity.
• The model suggests edits that align with known chemical principles: for example, it adds hydrophobic fragments to increase logP and hydrophilic groups to decrease it, even when no such examples were in the training data.
• Despite its success, the model occasionally generates unstable compounds (e.g., adjacent NH2 groups that would oxidize), highlighting the need for expert validation in the loop.
• The method enables fine-grained, expert-guided optimization through a select-and-replace interface that leverages chemists’ domain knowledge, offering a practical alternative to black-box generation.