[Paper Review] Universal Machine Learning Interatomic Potentials are Ready for Phonons

Antoine Loew, Dewen Sun+3 more|arXiv (Cornell University)|Dec 21, 2024

Machine Learning in Materials Science4 citations

TL;DR

This paper benchmarks seven universal machine learning interatomic potentials (uMLIPs) for phonon property prediction using ~10,000 ab initio phonon calculations. Despite strong performance in energy and force prediction at equilibrium, several models show significant inaccuracies in phonon frequencies and thermodynamic properties, highlighting the need to explicitly train uMLIPs for vibrational responses beyond ground-state properties.

ABSTRACT

There has been an ongoing race for the past several years to develop the best universal machinelearning interatomic potential. This progress has led to increasingly accurate models for predictingenergy, forces, and stresses, combining innovative architectures with big data. Here, we benchmarkthese models on their ability to predict harmonic phonon properties, which are critical for under-standing the vibrational and thermal behavior of materials. Using around 10 000 ab initio phononcalculations, we evaluate model performance across various phonon-related parameters to test theuniversal applicability of these models. The results reveal that some models achieve high accuracyin predicting harmonic phonon properties. However, others still exhibit substantial inaccuracies,even if they excel in the prediction of the energy and the forces for materials close to dynamicalequilibrium. These findings highlight the importance of considering phonon-related properties inthe development of universal machine learning interatomic potentials.

Motivation & Objective

• To evaluate the performance of state-of-the-art universal machine learning interatomic potentials (uMLIPs) in predicting phonon properties.
• To identify whether uMLIPs trained primarily on equilibrium geometries can accurately capture second derivatives of the potential energy surface, essential for phonons.
• To assess the trade-offs between predictive accuracy, computational efficiency, and generalization across diverse materials in phonon calculations.
• To provide a standardized phonon dataset and code to support future development and benchmarking of uMLIPs.

Proposed method

• The study uses the finite displacement method via the phonopy package to compute force constants and phonon dispersion from PBE-functional DFT calculations on 10,000 materials.
• All uMLIPs are evaluated using the same PBE-optimized structures, with geometry relaxation performed via ASE's FIRE algorithm and symmetry preservation via FretchCellFilter.
• Thermodynamic properties (vibrational entropy, Helmholtz free energy, heat capacity) are computed by Fourier interpolation of the phonon density of states on a 20×20×20 q-grid at 300 K.
• Models are tested on a diverse dataset of 10,000 materials from the MDR and MPtrj databases, with training data sizes ranging from 188k to 110M structures.
• The benchmark includes seven uMLIPs: M3GNet, MACE, CHGNet, MatterSim, SevenNet, ORB, and OMat24, each evaluated for energy, forces, phonon frequencies, and thermodynamic properties.
• All models are open-source, and the full phonon dataset is published in Alexandria under a Creative Commons Attribution 4.0 license.

Figure 1: Distribution of (a) number of different chemical elements per unit cell, (b) crystal systems, and (c) band gaps calculated with the PBE functional for all the materials in the dataset.

Experimental results

Research questions

• RQ1 Can universal machine learning interatomic potentials accurately predict phonon frequencies and thermodynamic properties across a broad range of materials?
• RQ2 Why do some uMLIPs with high accuracy in energy and force prediction still fail in phonon property prediction?
• RQ3 How do the computational efficiency and architectural design of uMLIPs correlate with their phonon prediction accuracy?
• RQ4 To what extent do off-equilibrium training data improve phonon property prediction in uMLIPs?

Key findings

• MatterSim and, to a lesser extent, SevenNet achieve the highest accuracy in phonon frequency prediction, with low dispersion and minimal systematic deviation from PBE reference values.
• Several models, including M3GNet and CHGNet, show significant errors in phonon frequencies despite strong performance in energy and force prediction at equilibrium.
• Models trained without explicit off-equilibrium data exhibit larger errors in phonon properties, indicating that curvature information is not sufficiently captured by standard training protocols.
• M3GNet is the fastest model, outperforming even GPU-accelerated models on a single CPU core, while OMat24 and MACE are the slowest among the tested uMLIPs.
• The study reveals that phonon properties are not reliably predicted by uMLIPs trained solely on equilibrium structures, underscoring the need to include curvature-sensitive data in future training.
• The authors release a comprehensive phonon dataset of ~10,000 materials and associated code to support future uMLIP development and benchmarking.

Figure 2: Periodic tables showing the frequency of the chemical elements in the structures from the dataset. Elements in gray are absent from the dataset.