[Paper Review] Gaussian Processes with Spectral Delta kernel for higher accurate Potential Energy surfaces for large molecules

Rodrigo A. Vargas–Hernández, Jake R. Gardner|arXiv (Cornell University)|Sep 28, 2021

Calibration and Measurement Techniques4 citations

TL;DR

This paper proposes a novel spectral delta kernel (k_SD) for Gaussian Processes (GPs) that leverages Bochners' theorem to enhance accuracy in interpolating high-dimensional potential energy surfaces (PESs). By optimizing the kernel via log-marginal likelihood and using GPU-accelerated GPyTorch and KEOPS, the method achieves sub-0.06 kcal/mol global error on large molecules like benzene and aspirin with only 15,000–20,000 training points, outperforming state-of-the-art models such as GDML and DTNN.

ABSTRACT

The interpolation of high-dimensional potential energy surfaces (PESs) is commonly done with physically-inspired deep-neural network models. In this work, we illustrate that Gaussian Processes (GPs) are also capable of interpolating high-dimensional complex physical systems. The accuracy of GPs depends on the robustness of the kernel function, and a boost in the accuracy is achieved by linearly combining kernel functions. In this work, we proposed an alternative route by parametrizing the kernel function through Bochners' theorem. We interpolated the PES of various chemical systems achieving a global accuracy of < 0.06 kcal/mol for Benzene, Malonaldehyde, Ethanol, and protonated Imidazole dimer using only 15 000 training points. Additionally, for Aspirin, we achieved a global error of 0.063 kcal/mol with 20 000 points. Given these results, we believe this kernel function is system-agnostic and could allow GPs to tackle a wider variety of high-dimensional physical systems.

Motivation & Objective

• To develop a more accurate and robust kernel function for Gaussian Processes in high-dimensional potential energy surface (PES) interpolation.
• To overcome the limitations of traditional kernel combination methods in scalability and accuracy for complex molecular systems.
• To enable full GP training without low-rank approximations (e.g., Nyström) by leveraging modern deep learning libraries and GPU acceleration.
• To demonstrate that GPs with the spectral delta kernel can achieve state-of-the-art accuracy comparable or superior to deep neural networks and gradient-domain models.
• To establish a system-agnostic, scalable, and uncertainty-quantifying framework for PES modeling in quantum chemistry.

Proposed method

• Proposes a spectral delta kernel (k_SD) derived from Bochners' theorem, parameterizing the kernel via a spectral density function.
• Uses the log-marginal likelihood as the objective function to optimize the kernel hyperparameters, including the spectral width |δ|.
• Employs GPyTorch and KEOPS libraries to enable efficient full GP training on GPUs, avoiding the Nyström approximation.
• Applies the kernel to high-dimensional PESs of molecules including benzene (61D), aspirin (57D), and protonated imidazole dimer (51D).
• Splits large PESs into fragments and trains separate GPs to improve global accuracy, particularly for high-dimensional systems.
• Validates performance using mean absolute error (MAE) and root mean square error (RMSE) against GDML and DTNN benchmarks.

Experimental results

Research questions

• RQ1 Can a kernel function derived from Bochners’ theorem significantly improve the accuracy of Gaussian Processes in interpolating high-dimensional potential energy surfaces?
• RQ2 How does the spectral delta kernel compare to standard kernel combinations and gradient-domain machine learning (GDML) in terms of predictive accuracy and data efficiency?
• RQ3 What is the optimal relationship between the number of training points and the spectral width |δ| for achieving minimal error in PES interpolation?
• RQ4 Can full Gaussian Process regression be efficiently trained on large molecular systems (e.g., 50+ dimensions) without low-rank covariance approximations?
• RQ5 To what extent can the spectral delta kernel generalize across diverse molecular systems, including those with high dimensionality and complex potential energy landscapes?

Key findings

• The spectral delta kernel (k_SD) achieved a global RMSE of 0.06 kcal/mol for benzene, malonaldehyde, ethanol, and protonated imidazole dimer using only 15,000 training points.
• For aspirin (57D), the method achieved a global MAE of 0.063 kcal/mol with 20,000 training points, outperforming GDML and DTNN.
• With 18,000 training points and |δ| = 5,000, the method achieved a MAE of 3.05 meV for malonaldehyde, 0.35 meV more accurate than GDML (3.3 meV).
• For benzene, a GP with |δ| ≥ 1,000 and N = 10,000 achieved MAE < 1 meV (~0.023 kcal/mol), demonstrating sub-kcal/mol accuracy.
• The optimal |δ| was found to lie in the range [N/3, N/2], with higher accuracy observed when |δ| increased beyond 5,000 for systems with >5,000 training points.
• The method enables accurate full GP training on 50+ dimensional PESs without Nyström approximation, using GPyTorch and KEOPS with GPU acceleration.