[Paper Review] GraphNVP: An Invertible Flow Model for Generating Molecular Graphs

Kaushalya Madhawa, Katushiko Ishiguro+2 more|arXiv (Cornell University)|May 28, 2019

Machine Learning in Materials ScienceMaterials Science86 citations

TL;DR

GraphNVP introduces an invertible normalizing-flow model that generates molecular graphs in two steps (adjacency tensor and node attributes) with exact likelihood training and high uniqueness of generated molecules. The latent space can be leveraged to optimize chemical properties.

ABSTRACT

We propose GraphNVP, the first invertible, normalizing flow-based molecular graph generation model. We decompose the generation of a graph into two steps: generation of (i) an adjacency tensor and (ii) node attributes. This decomposition yields the exact likelihood maximization on graph-structured data, combined with two novel reversible flows. We empirically demonstrate that our model efficiently generates valid molecular graphs with almost no duplicated molecules. In addition, we observe that the learned latent space can be used to generate molecules with desired chemical properties.

Motivation & Objective

• Motivate efficient generation of valid molecular graphs for drug discovery.
• Propose an invertible flow framework to model graph-structured data with two latent representations (adjacency tensor and node attributes).
• Achieve exact likelihood maximization for graphs and enable property-directed molecule generation.

Proposed method

• Decompose graph generation into two steps: generate an adjacency tensor and generate node attributes.
• Use two types of reversible affine coupling layers (adjacency coupling and node feature coupling) to transform A' and X' into latent representations z_A and z_X.
• Apply dequantization by adding uniform noise to discrete graph components before flow transformations.
• Train the model by maximizing the exact log-likelihood under a normalizing-flow composed of invertible transformations.
• During generation, sample latent vectors z, apply inverse coupling steps to obtain a probabilistic adjacency tensor, derive a discrete adjacency, then generate node features conditioned on the adjacency.

Experimental results

Research questions

• RQ1 Can an invertible flow model generate valid molecular graphs with high uniqueness without domain-specific validity scaffolding?
• RQ2 Does a two-step generation (adjacency then node attributes) improve validity and reconstruction performance for molecular graphs?
• RQ3 Is the learned latent space amenable to property-directed molecular optimization?
• RQ4 How does GraphNVP compare against VAEs and GANs on standard molecular graph benchmarks (QM9, ZINC-250k) in terms of validity, novelty, uniqueness, and reconstruction?

Key findings

• GraphNVP achieves high validity and reconstruction, and extremely high uniqueness with few duplicates among generated molecules.
• The model guarantees reconstruction accuracy due to invertible design (R = 100%).
• Latent-space manipulations can guide generation toward desirable chemical properties (e.g., QED) through simple linear or Bayesian methods.
• Latent-space interpolations around training molecules yield smooth transitions in generated structures, facilitating local molecular optimization.
• Two-step generation (structure then attributes) supports efficient and valid graph generation for molecules.