[Paper Review] M$^2$Hub: Unlocking the Potential of Machine Learning for Materials Discovery

Yuanqi Du, Yingheng Wang+7 more|arXiv (Cornell University)|Jun 14, 2023

Machine Learning in Materials ScienceMaterials Science3 citations

TL;DR

M²Hub is a comprehensive machine learning toolkit designed to accelerate materials discovery by integrating datasets, tasks, models, and benchmarks across virtual screening, inverse design, and molecular simulation. It provides 11 datasets spanning 6 material types and 56 tasks with realistic data splits, enabling benchmarking of state-of-the-art models—including those for non-periodic structures—on property prediction and generative design, with open-source code and transparent evaluation protocols.

ABSTRACT

We introduce M$^2$Hub, a toolkit for advancing machine learning in materials discovery. Machine learning has achieved remarkable progress in modeling molecular structures, especially biomolecules for drug discovery. However, the development of machine learning approaches for modeling materials structures lag behind, which is partly due to the lack of an integrated platform that enables access to diverse tasks for materials discovery. To bridge this gap, M$^2$Hub will enable easy access to materials discovery tasks, datasets, machine learning methods, evaluations, and benchmark results that cover the entire workflow. Specifically, the first release of M$^2$Hub focuses on three key stages in materials discovery: virtual screening, inverse design, and molecular simulation, including 9 datasets that covers 6 types of materials with 56 tasks across 8 types of material properties. We further provide 2 synthetic datasets for the purpose of generative tasks on materials. In addition to random data splits, we also provide 3 additional data partitions to reflect the real-world materials discovery scenarios. State-of-the-art machine learning methods (including those are suitable for materials structures but never compared in the literature) are benchmarked on representative tasks. Our codes and library are publicly available at https://github.com/yuanqidu/M2Hub.

Motivation & Objective

• Address the gap in machine learning infrastructure for materials discovery compared to molecular and biomolecular applications.
• Provide a centralized, integrated platform that supports the full ML workflow in materials science: data, tasks, models, evaluation, and benchmarking.
• Enable realistic evaluation through multiple data splits, including out-of-distribution scenarios, to improve model robustness.
• Support generative materials design by providing synthetic datasets, evaluation metrics, and oracle functions.
• Benchmark diverse machine learning models—including those not previously compared in materials contexts—on representative tasks.

Proposed method

• Curate 11 datasets across 6 material types (e.g., crystals, 2D materials, MOFs) covering 56 tasks across 8 material properties.
• Design three additional data splits beyond random splits to reflect real-world out-of-distribution scenarios in materials discovery.
• Integrate 13 representative machine learning models, including graph neural networks (CGCNN, MEGNet, SchNet, DimeNet++, GemNet, Equiformer, LEFTNet) and composition-based models (ALIGNN), for property prediction and generative tasks.
• Implement standardized data processing pipelines and evaluation protocols, including metrics like MAE, RMSE, and R² for regression tasks.
• Provide oracle functions and evaluation frameworks for inverse design and generative materials modeling.
• Host all code, datasets, and benchmark results in a public GitHub repository for reproducibility and community use.

Experimental results

Research questions

• RQ1 How can a unified platform improve the reproducibility and benchmarking of machine learning models in materials discovery?
• RQ2 What are the performance differences of state-of-the-art models across diverse materials and property prediction tasks, including non-periodic structures?
• RQ3 How do realistic data splits (e.g., out-of-distribution) affect model generalization and evaluation reliability?
• RQ4 To what extent can existing machine learning models be effectively applied to generative materials design tasks?
• RQ5 What role do model architecture, training scheme, and objective function play in solving specific materials discovery problems?

Key findings

• M²Hub provides 11 curated datasets with 56 tasks across 8 material properties, including 2 synthetic datasets for generative modeling.
• The benchmark includes three realistic data splits beyond random splits, enhancing model evaluation robustness in out-of-distribution settings.
• Among tested models, DimeNet++ and GemNet achieved the best average ranking (2.4 and 2.9) on the qmof bandgap task, while ALIGNN showed strong performance on composition-based tasks with an average ranking of 2.3.
• Training efficiency varies significantly: CGCNN and SchNet are faster (e.g., 68s for pdos on V100), while Equiformer and LEFTNet are slower (e.g., 713s and 117s for pdos), indicating trade-offs between accuracy and inference speed.
• Models like EGNN and DimeNet++ show strong performance on multiple tasks, with EGNN achieving the lowest MAE (0.0113) on e_form under 100-atom systems.
• The benchmark reveals that model choice significantly impacts performance, with ALIGNN and SchNet showing strong generalization across diverse materials and property types.