[Paper Review] Compressing physical properties of atomic species for improving predictive chemistry

John E. Herr, Kevin J. Koh+2 more|arXiv (Cornell University)|Oct 31, 2018

Machine Learning in Materials ScienceMaterials Science78 references29 citations

TL;DR

This paper introduces 'elemental modes'—a compressed, low-dimensional representation of atomic species derived from fundamental physical properties using an auto-encoder. The method improves machine learning predictions in chemistry by encoding periodic trends and atomic similarities, outperforming standard feature representations in formation energy prediction and enabling transfer learning across elements with higher accuracy and generalization.

ABSTRACT

The answers to many unsolved problems lie in the intractable chemical space of molecules and materials. Machine learning techniques are rapidly growing in popularity as a way to compress and explore chemical space efficiently. One of the most important aspects of machine learning techniques is representation through the feature vector, which should contain the most important descriptors necessary to make accurate predictions, not least of which is the atomic species in the molecule or material. In this work we introduce a compressed representation of physical properties for atomic species we call the elemental modes. The elemental modes provide an excellent representation by capturing many of the nuances of the periodic table and the similarity of atomic species. We apply the elemental modes to several different tasks for machine learning algorithms and show that they enable us to make improvements to these tasks even beyond simply achieving higher accuracy predictions.

Motivation & Objective

• To address the challenge of representing atomic species in machine learning models for predictive chemistry.
• To develop a generalizable, low-dimensional representation that captures periodic trends and atomic similarities.
• To improve machine learning performance in materials property prediction by encoding atomic species more effectively.
• To enable transfer learning across elements by sharing information through a shared, continuous representation.
• To avoid dataset bias by deriving the representation from fundamental physical properties rather than atomic environments.

Proposed method

• An auto-encoder is trained on 10 fundamental physical properties (e.g., atomic number, electronegativity, ionization energy) for 83 elements (Z ≤ 83, excluding f-block).
• The encoder produces a 6-dimensional latent vector per element, termed 'elemental modes,' optimized for compression and reconstruction fidelity.
• The representation is derived from intrinsic atomic properties, avoiding bias from specific materials datasets.
• Principal component analysis of the elemental modes reveals strong alignment with periodic table trends, such as alkali metals and halogens forming distinct clusters.
• The elemental modes are used as input features in a feed-forward neural network for formation energy prediction on elpasolite materials.
• The method enables transfer learning by allowing similar elements to share information through a continuous, shared embedding space.

Experimental results

Research questions

• RQ1 Can a compressed, continuous representation of atomic species improve machine learning performance in predictive chemistry?
• RQ2 Does the elemental modes representation capture meaningful periodic trends and atomic similarities more effectively than discrete or scalar features?
• RQ3 Can elemental modes enable better generalization and transfer learning across different atomic species in materials property prediction?
• RQ4 How does the elemental modes representation compare to existing methods (e.g., atomic number, group/period, learned embeddings) in accuracy and robustness?
• RQ5 To what extent does the method generalize across distinct chemical tasks beyond formation energy prediction?

Key findings

• The elemental modes representation achieved a mean absolute error (MAE) of 0.086 eV/atom in predicting formation energies for elpasolite materials, outperforming prior methods.
• The method demonstrated improved generalization by enabling transfer learning across elements, as similar atoms shared information through the continuous embedding space.
• Principal component analysis of the elemental modes revealed clear, interpretable clustering by periodic table groups and periods, confirming the method captures known chemical trends.
• The 6-dimensional elemental modes representation provided better compression and reconstruction of physical properties than higher-dimensional or scalar alternatives.
• The approach generalized beyond the elpasolite dataset, showing success in other tasks such as predicting partial charges and bond energies.
• The auto-encoder-based method avoided dataset bias by relying on fundamental physical properties rather than atomic environment statistics.