[Paper Review] Benchmarking of machine learning interatomic potentials for reactive hydrogen dynamics at metal surfaces

Wojciech G. Stark, van der Oord+5 more|arXiv (Cornell University)|Mar 22, 2024

Machine Learning in Materials Science4 citations

TL;DR

This study benchmarks state-of-the-art machine learning interatomic potentials—PaiNN, REANN, MACE, and ACE—for simulating reactive hydrogen dynamics on copper surfaces. REANN and MACE deliver the best balance of accuracy and inference speed (0.2–0.5 ms/atom/step on CPU), enabling efficient high-throughput simulation of sticking probabilities, while ACE offers the fastest inference but requires improved training data for consistent accuracy.

ABSTRACT

Simulations of chemical reaction probabilities in gas surface dynamics require the calculation of ensemble averages over many tens of thousands of reaction events to predict dynamical observables that can be compared to experiments. At the same time, the energy landscapes need to be accurately mapped, as small errors in barriers can lead to large deviations in reaction probabilities. This brings a particularly interesting challenge for machine learning interatomic potentials, which are becoming well-established tools to accelerate molecular dynamics simulations. We compare state-of-the-art machine learning interatomic potentials with a particular focus on their inference performance on CPUs and suitability for high throughput simulation of reactive chemistry at surfaces. The considered models include polarizable atom interaction neural networks (PaiNN), recursively embedded atom neural networks (REANN), the MACE equivariant graph neural network, and atomic cluster expansion potentials (ACE). The models are applied to a dataset on reactive molecular hydrogen scattering on low-index surface facets of copper. All models are assessed for their accuracy, time-to-solution, and ability to simulate reactive sticking probabilities as a function of the rovibrational initial state and kinetic incidence energy of the molecule. REANN and MACE models provide the best balance between accuracy and time-to-solution and can be considered the current state-of-the-art in gas-surface dynamics. PaiNN models require many features for the best accuracy, which causes significant losses in computational efficiency. ACE models provide the fastest time-to-solution, however, models trained on the existing dataset were not able to achieve sufficiently accurate predictions in all cases.

Motivation & Objective

• To evaluate the performance of modern machine learning interatomic potentials (MLIPs) in simulating reactive hydrogen scattering on copper surfaces.
• To assess accuracy, inference speed, and suitability for high-throughput molecular dynamics simulations of reaction probabilities.
• To identify the most efficient and accurate MLIPs for modeling complex surface reaction dynamics with statistical sampling requirements.
• To compare the impact of different architectural designs—message-passing neural networks (PaiNN, REANN, MACE) versus atomic cluster expansion (ACE)—on predictive performance.

Proposed method

• The study employs a dataset of reactive H2 scattering on low-index Cu facets generated via active learning with an initial MPNN-based MLIP.
• Five MLIPs are trained: PaiNN, REANN, MACE (equivariant graph neural network), and ACE potentials using the ACEPotentials.jl framework.
• All models are evaluated for accuracy in predicting energy landscapes, phonon band structures, and lattice expansion across a range of temperatures.
• Inference performance is measured in time-to-solution per force evaluation per atom on CPUs, with REANN and MACE achieving 0.2 and 0.5 ms/atom/step, respectively.
• Reactive sticking probabilities are computed as ensemble averages over tens of thousands of trajectories, comparing MLIP predictions to reference data.
• Hyperparameter optimization is performed to balance accuracy and computational efficiency across all models.

Experimental results

Research questions

• RQ1 Which machine learning interatomic potential offers the best trade-off between accuracy and inference speed for simulating reactive hydrogen dynamics on metal surfaces?
• RQ2 How do message-passing neural networks (PaiNN, REANN, MACE) compare to atomic cluster expansion (ACE) potentials in predicting reaction probabilities for H2 on Cu?
• RQ3 Can ACE potentials achieve comparable accuracy to neural network-based models with significantly faster inference times when trained on the same dataset?
• RQ4 To what extent does model architecture influence the ability to capture complex reaction pathways and energy barriers in gas-surface dynamics?
• RQ5 What role does training data quality and curation play in the predictive performance of ACE-based MLIPs compared to deep learning-based approaches?

Key findings

• REANN and MACE models achieve the best balance between accuracy and inference speed, with time-to-solution of 0.2 ms and 0.5 ms per force evaluation per atom on CPUs, respectively.
• PaiNN models require many features for high accuracy, resulting in significant computational overhead and reduced efficiency despite strong predictive performance.
• ACE potentials deliver the fastest inference time (~0.1 ms/atom/step), but failed to achieve consistent accuracy across all reaction coordinates with the given training data.
• All models accurately predict lattice expansion and phonon band structures, indicating robustness in describing bulk and surface properties of copper.
• With optimized hyperparameters, MACE and REANN reproduce experimental and reference simulation results for H2 sticking probabilities as a function of kinetic energy and vibrational state.
• The study suggests that ACE potentials may require higher-quality or more balanced training data to match the accuracy of neural network-based models.