[Paper Review] Relevance of Rotationally Equivariant Convolutions for Predicting Molecular Properties

B. Miller, Mario Geiger+2 more|arXiv (Cornell University)|Aug 19, 2020

Machine Learning in Materials Science27 references45 citations

TL;DR

The paper compares rotationally invariant versus equivariant neural network layers (L0Nets vs L1Nets) in e3nn for QM9, finding angular features reduce test error by 23% on average at fixed depth, while increasing depth yields only 4% improvement.

ABSTRACT

Equivariant neural networks (ENNs) are graph neural networks embedded in $\mathbb{R}^3$ and are well suited for predicting molecular properties. The ENN library e3nn has customizable convolutions, which can be designed to depend only on distances between points, or also on angular features, making them rotationally invariant, or equivariant, respectively. This paper studies the practical value of including angular dependencies for molecular property prediction directly via an ablation study with exttt{e3nn} and the QM9 data set. We find that, for fixed network depth and parameter count, adding angular features decreased test error by an average of 23%. Meanwhile, increasing network depth decreased test error by only 4% on average, implying that rotationally equivariant layers are comparatively parameter efficient. We present an explanation of the accuracy improvement on the dipole moment, the target which benefited most from the introduction of angular features.

Motivation & Objective

• Evaluate whether angular (equivariant) features in SE(3) networks improve molecular property prediction over distance-only (invariant) features.
• Quantify performance gains at fixed model capacity between L0Nets and L1Nets on QM9.
• Understand how depth vs angularity contributes to predictive accuracy in ENNs for molecular data.

Proposed method

• Use e3nn library with SE(3) equivariant point modules to build L0Net (invariant) and L1Net (equivariant) architectures.
• Represent molecules as sets of atoms with positions and element features, enforcing rotation, translation, and permutation invariance.
• Compare models that differ in whether internal features rotate (u1 != 0) or do not rotate (u1 = 0).
• Train and evaluate on QM9 scalar properties, analyzing average test error differences at fixed depth and parameter count.
• Express layer updates with F^h = vec(F^h_ell=0 ⊕ F^h_ell=1) and analyze rotation actions via R on features (block matrix form).
• Interpret results with focus on dipole moment performance as the target showing notable gains from angular features.]
• research_questions([
• What is the benefit, if any, of including angular features in SE(3) equivariant networks for predicting scalar molecular properties compared to purely distance-based invariant layers?
• How do depth and angular features interact to affect predictive accuracy on QM9?

Experimental results

Research questions

• RQ1 What is the benefit, if any, of including angular features in SE(3) equivariant networks for predicting scalar molecular properties compared to purely distance-based invariant layers?
• RQ2 How do depth and angular features interact to affect predictive accuracy on QM9?

Key findings

• Angular features reduce test error by an average of 23% at fixed depth and parameter count.
• Increasing network depth reduces test error by only 4% on average.
• Rotationally equivariant layers are comparatively parameter efficient.
• Dipole moment prediction benefits most from the introduction of angular features.
• L0Nets (invariant layers) vs L1Nets (equivariant layers) differ in whether internal features rotate, impacting performance.
• The study situates ENNs within QM9, highlighting the value of angular information for molecular property regression.