[Paper Review] Crystal Structure Representations for Machine Learning Models of Formation Energies

Felix A. Faber, Alexander Lindmaa+2 more|arXiv (Cornell University)|Jan 1, 2015

Machine Learning in Materials Science3 references18 citations

TL;DR

This paper proposes and evaluates three crystal structure representations—Ewald sum matrix, extended Coulomb matrix, and sine matrix—for machine learning models predicting formation energies of solids. Using kernel ridge regression on a 3,938-structure Materials Project dataset, the sine matrix representation achieves the lowest generalization error of 0.37 eV/atom, outperforming the Ewald (0.49 eV/atom) and extended Coulomb (0.64 eV/atom) methods while being computationally most efficient.

ABSTRACT

We introduce and evaluate a set of feature vector representations of crystal structures for machine learning (ML) models of formation energies of solids. ML models of atomization energies of organic molecules have been successful using a Coulomb matrix representation of the molecule. We consider three ways to generalize such representations to periodic systems: (i) a matrix where each element is related to the Ewald sum of the electrostatic interaction between two different atoms in the unit cell repeated over the lattice; (ii) an extended Coulomb-like matrix that takes into account a number of neighboring unit cells; and (iii) an Ansatz that mimics the periodicity and the basic features of the elements in the Ewald sum matrix by using a sine function of the crystal coordinates of the atoms. The representations are compared for a Laplacian kernel with Manhattan norm, trained to reproduce formation energies using a data set of 3938 crystal structures obtained from the Materials Project. For training sets consisting of 3000 crystals, the generalization error in predicting formation energies of new structures corresponds to (i) 0.49, (ii) 0.64, and (iii) 0.37 eV/atom for the respective representations.

Motivation & Objective

• To generalize successful molecular machine learning representations, such as the Coulomb matrix, to periodic crystal systems.
• To evaluate the performance of three distinct crystal structure representations in predicting formation energies using machine learning.
• To identify the most efficient and accurate representation for modeling formation energies in solids.
• To assess the impact of training set size and data diversity on model generalization error.
• To explore the feasibility of achieving near-first-principles accuracy in formation energy prediction for periodic systems.

Proposed method

• The Ewald sum matrix representation computes electrostatic interactions between atoms in a unit cell using the Ewald summation technique, accounting for periodic boundary conditions.
• The extended Coulomb matrix representation extends the Coulomb matrix concept by including interactions with neighboring unit cells to capture long-range periodic effects.
• The sine matrix representation approximates the periodic features of the Ewald sum using a sine function of atomic coordinates, reducing computational cost while preserving key structural characteristics.
• Kernel ridge regression with a Laplacian kernel and Manhattan norm is used to train models on formation energy data from the Materials Project.
• Principal component analysis (PCA) is applied to visualize the structure of the feature representations and assess their clustering and separability.
• Generalization error is evaluated using cross-validation on training sets of varying sizes, with performance measured in eV/atom.

Experimental results

Research questions

• RQ1 Can the Coulomb matrix representation used successfully for molecules be effectively generalized to periodic crystals for formation energy prediction?
• RQ2 How do different crystal structure representations—Ewald sum, extended Coulomb, and sine matrix—compare in terms of predictive accuracy and computational cost?
• RQ3 Does the generalization error decrease with increasing training set size, and what does this imply for model scalability?
• RQ4 Why is the performance of ML models for solids significantly worse than for molecules, despite similar methods?
• RQ5 Can a more compact, chemically restricted data subset improve model accuracy and reduce the need for large training sets?

Key findings

• The sine matrix representation achieves the lowest generalization error of 0.37 eV/atom on a 3,000-structure training set, outperforming the Ewald sum matrix (0.49 eV/atom) and extended Coulomb matrix (0.64 eV/atom).
• All three representations show a systematic decrease in generalization error with increasing training set size, indicating potential for further improvement with larger datasets.
• The sine matrix representation is not only the most accurate but also the most computationally efficient, making it the most practical choice for large-scale applications.
• Principal component analysis reveals distinct clustering patterns: the Ewald sum matrix shows strong clustering and outliers, while the sine and extended Coulomb matrices are more similar in structure.
• The performance gap between molecular and solid systems is attributed to the larger chemical and structural diversity in the Materials Project dataset compared to the QM7 molecular dataset.
• The results suggest that with larger or more restricted training data, machine learning models for solid formation energies could achieve near-first-principles accuracy.