[Paper Review] Improved uncertainty quantification for Gaussian process regression based interatomic potentials

Albert P. Bartók, James. R. Kermode|arXiv (Cornell University)|Jun 17, 2022

Machine Learning in Materials Science4 citations

TL;DR

This paper proposes optimizing hyperparameters in Gaussian process regression (GPR)-based interatomic potentials using leave-one-out cross-validation (LOO-CV) likelihood instead of marginal likelihood, significantly improving uncertainty quantification (UQ) reliability. The LOO-CV approach reduces overestimation of prediction errors—demonstrated on Ar trimer and dimer systems—yielding more accurate and robust error estimates than standard GPR or heuristic tuning.

ABSTRACT

The error estimation capability of machine learning interatomic potentials (MLIPs) based on probabilistic learning methods such as Gaussian process regression (GPR) is currently under-exploited, because of the tendancy of the predicted errors to overestimate the true error. We present approaches based on maximising either the marginal likelihood or an alternative likelihood constructed using leave-one-out cross validation to provide improved error estimates for interatomic potentials based on GPR. We benchmarked these approaches on models representing the Ar trimer, showing significant improvements in the robustness of the predicted error estimates.

Motivation & Objective

• Address the underutilization of uncertainty quantification (UQ) in machine learning interatomic potentials (MLIPs), particularly the overestimation of prediction errors by Gaussian process regression (GPR).
• Overcome the limitations of marginal likelihood maximization, which often leads to overconfident or inaccurate error estimates in GPR-based MLIPs.
• Develop a more robust alternative to heuristic hyperparameter selection in Gaussian approximate potentials (GAP) by using LOO-CV likelihood optimization.
• Improve the reliability of predicted error bars for derived quantities of interest (QoIs) in multiscale materials modeling, enabling better uncertainty propagation across scales.
• Demonstrate that LOO-CV-based optimization yields more accurate and realistic uncertainty estimates than standard GPR or heuristic approaches on small, ab initio benchmark systems.

Proposed method

• Optimize hyperparameters of GPR-based interatomic potentials by maximizing the leave-one-out cross-validation (LOO-CV) likelihood, which estimates predictive performance without assuming model correctness.
• Compare LOO-CV optimization against standard marginal likelihood maximization and heuristic hyperparameter selection (GAP heuristics) in a two- and three-body Ar system.
• Use explicit basis functions to model short-range repulsion and long-range dispersion, with kernel hyperparameters (e.g., lengthscale ℓ, signal variance δ) optimized via BFGS with automatic differentiation.
• Evaluate model performance using root mean square error (RMSE) and compare predicted error variances to true errors from cross-validation.
• Decompose uncertainty contributions into parametric and non-parametric terms to analyze how hyperparameter choices affect error structure.
• Apply Pareto-smoothed importance sampling (PSIS) as a scalable alternative for future extension to large datasets and sparse GP settings.

Experimental results

Research questions

• RQ1 Can LOO-CV likelihood optimization produce more reliable uncertainty estimates than marginal likelihood maximization in GPR-based interatomic potentials?
• RQ2 How does the LOO-CV approach compare to heuristic hyperparameter tuning in terms of error estimation accuracy and robustness?
• RQ3 What impact do different hyperparameter choices (e.g., δ, ℓ) have on the decomposition of uncertainty in two- and three-body terms?
• RQ4 Does LOO-CV optimization reduce the tendency of GPR to overestimate prediction errors, especially in regions far from training data?
• RQ5 Can the LOO-CV approach be scaled to large datasets and complex many-body descriptors like SOAP or ACE in practical materials simulations?

Key findings

• The LOO-CV approach significantly improves the reliability of predicted error estimates compared to both marginal likelihood maximization and heuristic hyperparameter selection.
• While marginal likelihood optimization achieved the lowest RMSE (1.7 meV/atom), it still overestimated true errors in some regions, indicating poor calibration of uncertainty.
• The LOO-CV method produced the most accurate error estimates, with a lower RMSE of 2.4 meV/atom but far better correlation between predicted and true errors.
• LOO-CV led to a more balanced weighting of two- and three-body terms, with δ₂ = 1429 eV and δ₃ = 0.0006 eV, indicating stronger emphasis on two-body interactions.
• The lengthscale parameters in the LOO-CV model were more evenly distributed across three-body descriptor components (ℓ₁ = 239 Å, ℓ₂ = 329 Ų, ℓ₃ = 344 Å), suggesting better generalization.
• The explicit basis function approach contributed significantly to uncertainty at long distances, and this contribution was larger in the LOO-CV case, indicating improved modeling of out-of-distribution behavior.