[Paper Review] Cormorant: Covariant Molecular Neural Networks

Brandon Anderson, Truong-Son Hy+1 more|arXiv (Cornell University)|Jun 6, 2019

Machine Learning in Materials Science34 references142 citations

TL;DR

Cormorant is a rotationally covariant neural network for learning molecular potentials and ground-state properties, using SO(3) covariant activations and Clebsch–Gordan nonlinearities to achieve state-of-the-art results on MD-17 and competitive performance on QM9.

ABSTRACT

We propose Cormorant, a rotationally covariant neural network architecture for learning the behavior and properties of complex many-body physical systems. We apply these networks to molecular systems with two goals: learning atomic potential energy surfaces for use in Molecular Dynamics simulations, and learning ground state properties of molecules calculated by Density Functional Theory. Some of the key features of our network are that (a) each neuron explicitly corresponds to a subset of atoms; (b) the activation of each neuron is covariant to rotations, ensuring that overall the network is fully rotationally invariant. Furthermore, the non-linearity in our network is based upon tensor products and the Clebsch-Gordan decomposition, allowing the network to operate entirely in Fourier space. Cormorant significantly outperforms competing algorithms in learning molecular Potential Energy Surfaces from conformational geometries in the MD-17 dataset, and is competitive with other methods at learning geometric, energetic, electronic, and thermodynamic properties of molecules on the GDB-9 dataset.

Motivation & Objective

• Motivate learning of molecular interactions with symmetry-aware architectures to improve accuracy and physical plausibility.
• Introduce Cormorant, a covariant neural network where neurons correspond to subsets of atoms and activations transform covariantly under rotations.
• Demonstrate that enforcing SO(3) covariance improves learning of potential energy surfaces and ground-state molecular properties.

Proposed method

• Represent activations as SO(3) vectors (scalar and higher-order tensors) to ensure rotation covariance.
• Use Clebsch–Gordan (CG) transforms as the nonlinear mixing operation to couple tensor components across layers.
• Structure neurons as one-body (atom) and two-body (pair) interactions with radial and distance-based gating.
• Train a three-part network: input featurization, CG-Nav (covariant) layers, and a rotation-invariant output head.
• Allow each neuron to output a fixed-type SO(3) vector, enabling learned interactions that resemble physical multipole forms.

Experimental results

Research questions

• RQ1 How can neural networks be designed to be covariant to rotations while learning molecular interactions?
• RQ2 Can a covariant architecture with CG nonlinearities outperform existing ML force fields and predict ground-state molecular properties?
• RQ3 What is the impact of representing activations as SO(3) tensors on learning accuracy for MD and DFT-derived targets?

Key findings

• Cormorant outperforms competing models on MD-17 for learning molecular potential energy surfaces.
• On QM9, Cormorant achieves leading or competitive results on several targets (α, Δε, εHOMO, εLUMO, μ, Cv).
• Across QM9, 12 targets show six with leading or competitive performance, and others within 40% of best (R^2 notably lower).
• For MD-17, Cormorant achieves superior MAE across evaluated molecules compared with DeepMD, DTNN, SchNet, GDML, and sGDML under comparable training settings.