[Paper Review] Semiempirical Hamiltonians learned from data can have accuracy comparable to Density Functional Theory

Frank Hu, Hanqing He+1 more|arXiv (Cornell University)|Oct 21, 2022

Machine Learning in Materials Science4 citations

TL;DR

This paper introduces DFTBML, a machine learning-enhanced semiempirical quantum chemistry model based on Density Functional Tight Binding (DFTB) with learnable one-dimensional functions of interatomic distance. By training these functions via data-driven optimization while preserving a physically interpretable form, the model achieves CCSD(T)*/CBS-level accuracy (within ~3 kcal/mol) at a fraction of the computational cost, outperforming traditional semiempirical methods and rivaling DFT with significantly reduced data requirements.

ABSTRACT

Quantum chemistry provides chemists with invaluable information, but the high computational cost limits the size and type of systems that can be studied. Machine learning (ML) has emerged as a means to dramatically lower cost while maintaining high accuracy. However, ML models often sacrifice interpretability by using components, such as the artificial neural networks of deep learning, that function as black boxes. These components impart the flexibility needed to learn from large volumes of data but make it difficult to gain insight into the physical or chemical basis for the predictions. Here, we demonstrate that semiempirical quantum chemical (SEQC) models can learn from large volumes of data without sacrificing interpretability. The SEQC model is that of Density Functional based Tight Binding (DFTB) with fixed atomic orbital energies and interactions that are one-dimensional functions of interatomic distance. This model is trained to ab initio data in a manner that is analogous to that used to train deep learning models. Using benchmarks that reflect the accuracy of the training data, we show that the resulting model maintains a physically reasonable functional form while achieving an accuracy, relative to coupled cluster energies with a complete basis set extrapolation (CCSD(T)*/CBS), that is comparable to that of density functional theory (DFT). This suggests that trained SEQC models can achieve low computational cost and high accuracy without sacrificing interpretability. Use of a physically-motivated model form also substantially reduces the amount of ab initio data needed to train the model compared to that required for deep learning models.

Motivation & Objective

• To develop a semiempirical quantum chemistry model that maintains physical interpretability while achieving high accuracy comparable to DFT.
• To reduce the data requirements typical of deep learning models by embedding physical constraints into the model architecture.
• To demonstrate that learnable parameters in a physically motivated Hamiltonian (DFTB) can achieve chemical accuracy without sacrificing interpretability.
• To show that a small dataset (~23,000 configurations) is sufficient to train a model that reaches performance saturation, limited by model form rather than data size.

Proposed method

• The model uses the DFTB Hamiltonian with fixed atomic orbital energies and one-dimensional functions of interatomic distance for interactions and overlaps, parameterized via high-order (fifth-degree) splines with 100 knots.
• A strong regularization scheme is applied to prevent non-physical oscillations in the spline functions during training.
• The model is trained using the ANI-1CCX dataset of ab initio energies for C, H, N, O molecules with up to eight heavy atoms, using a loss function combining total energy, dipole, and charge predictions with weighted loss terms.
• Backpropagation is enabled through loop inversion: the SCF and training loops are restructured so that charge fluctuations and model updates are computed outside the gradient descent path, enabling efficient differentiable SCF.
• The repulsive potential is updated every 10 epochs using convex optimization, ensuring global convergence.
• A linear reference energy correction is applied to atomization energies to enable fair comparison with CCSD(T)*/CBS, with coefficients derived via least-squares fitting.

Experimental results

Research questions

• RQ1 Can a semiempirical quantum chemistry model with a physically motivated form achieve accuracy comparable to DFT when trained on ab initio data?
• RQ2 Does using learnable spline functions in a DFTB framework reduce data requirements compared to deep learning models while maintaining high accuracy?
• RQ3 To what extent does preserving a physical Hamiltonian structure improve interpretability without compromising predictive performance?
• RQ4 Is the performance of the model limited by the model form or by the amount of training data?

Key findings

• The DFTBML model achieves a mean absolute error (MAE) of approximately 3 kcal/mol relative to CCSD(T)*/CBS reference data, matching the accuracy of standard DFT functionals.
• Performance saturates at around 20,000 molecular configurations, indicating that the model's accuracy is limited by its functional form rather than data quantity.
• The model requires only ~232,000 configurations (vs. ~1 million for deep learning models) to reach optimal performance, significantly reducing data needs.
• Outlier rates are below 0.05% for total energy predictions, indicating robustness and stability in generalization.
• The use of convex optimization for the repulsive potential ensures global convergence and stable training.
• The model maintains physical reasonableness in functional form while achieving high sensitivity to interatomic interactions through high-order splines with regularization.