[Paper Review] Global descriptors of a water molecule for machine learning of potential energy surfaces

Fabio E. A. Albertani, Alex J. W. Thom|arXiv (Cornell University)|Jan 10, 2023

Machine Learning in Materials ScienceMaterials Science3 citations

TL;DR

This paper investigates how global molecular descriptors influence Gaussian process (GP) regression for machine learning of potential energy surfaces (PES) in water molecules, focusing on permutational invariance. It demonstrates that symmetrizing training data through hydrogen permutation enhances GP performance on non-invariant feature spaces, reducing mean absolute error (MAE) from 7.5 to 1.9 mHa when using non-local normal mode coefficients.

ABSTRACT

Machine learning of multi-dimensional potential energy surfaces, from purely ab initio datasets, has seen substantial progress in the past years. Gaussian processes, a popular regression method, have been very successful at producing realistic potential energy surfaces from sparse datasets allowing to reduce the computational cost of highly accurate models. However, there are many choices one has to take in the design of the kernel and the feature space which can substantially change the performance and the characteristic of the model. In this study we explore different Gaussian processes set ups and, more specifically, how the permutational invariance of the target surface is controlled through the feature space selection.

Motivation & Objective

• To evaluate the impact of feature space selection on Gaussian process regression for potential energy surfaces.
• To investigate how permutational invariance—critical for molecules like water—is best enforced in GP models.
• To compare performance of different global descriptors (interatomic distances, normal modes, non-local normal modes) in learning symmetric PESs.
• To assess the effectiveness of data symmetrization (via permutation augmentation) in improving GP generalization without explicit invariance in the feature space.
• To examine the role of log-marginal likelihood maximization and hyperparameter optimization in GP model selection for sparse datasets.

Proposed method

• Uses Gaussian process regression with Matérn and RBF kernels to model the water molecule's PES from ab initio data.
• Employs global descriptors based on interatomic distances (MID), normal mode coefficients (NM), and non-local normal mode coefficients (NLNM) as input features.
• Applies data augmentation by permuting hydrogen atoms to generate symmetric training sets, even when the feature space lacks explicit permutational invariance.
• Symmetrizes feature representations by replacing mode coefficients with their signed permutations (e.g., [v1, v2, -v3]) to enforce invariance.
• Optimizes GP hyperparameters via log-marginal likelihood (LML) maximization, analyzing multiple local minima to avoid suboptimal solutions.
• Evaluates model performance using mean absolute error (MAE) on test sets and visualizes latent PESs through isovalue contours.

Experimental results

Research questions

• RQ1 How does the choice of global descriptor (MID, NM, NLNM) affect GP regression performance on water’s PES?
• RQ2 Can data augmentation via hydrogen permutation improve GP models trained on non-permutationally invariant feature spaces?
• RQ3 What is the impact of symmetrizing feature representations (e.g., [v1, v2, -v3])) on the learned PES symmetry and accuracy?
• RQ4 How does LML maximization influence model selection when multiple local minima exist in the hyperparameter space?
• RQ5 To what extent does symmetrized training data improve generalization on sparse datasets compared to unsymmetrized counterparts?

Key findings

• Symmetrizing the training data via hydrogen permutation reduced the MAE from 7.5 mHa to 1.9 mHa when using the non-local normal mode (NLNM) feature space.
• The latent GP surface trained on symmetrized NLNM data restored the expected permutational symmetry of the PES, as evidenced by symmetric isovalue contours.
• Even with non-invariant feature spaces, symmetrized data led to models that were more accurate and symmetric than unsymmetrized counterparts.
• The NLNM feature space outperformed the local NM space, achieving an MAE of 1.9 mHa compared to 7.5 mHa, indicating improved representation of PES complexity.
• LML maximization revealed multiple local minima, suggesting that Bayesian scoring alone may not reliably predict model performance, and alternative metrics are needed for evaluation.
• The study confirms that feature space design and data augmentation are critical for GP success in PES modeling, especially under data sparsity.