[Paper Review] Neural Networks for Constitutive Modeling -- From Universal Function Approximators to Advanced Models and the Integration of Physics

Johannes Dornheim, Lukas Morand+2 more|arXiv (Cornell University)|Feb 28, 2023

Drilling and Well EngineeringEngineering3 citations

TL;DR

This paper reviews neural network-based constitutive modeling in materials science, categorizing approaches from universal function approximators to physics-integrated models. It highlights how machine learning enables fast, accurate surrogate models for complex material behavior while addressing challenges like extrapolation, interpretability, and error accumulation through physical constraints and advanced architectures.

ABSTRACT

Analyzing and modeling the constitutive behavior of materials is a core area in materials sciences and a prerequisite for conducting numerical simulations in which the material behavior plays a central role. Constitutive models have been developed since the beginning of the 19th century and are still under constant development. Besides physics-motivated and phenomenological models, during the last decades, the field of constitutive modeling was enriched by the development of machine learning-based constitutive models, especially by using neural networks. The latter is the focus of the present review, which aims to give an overview of neural networks-based constitutive models from a methodical perspective. The review summarizes and compares numerous conceptually different neural networks-based approaches for constitutive modeling including neural networks used as universal function approximators, advanced neural network models and neural network approaches with integrated physical knowledge. The upcoming of these methods is in-turn closely related to advances in the area of computer sciences, what further adds a chronological aspect to this review. We conclude this review paper with important challenges in the field of learning constitutive relations that need to be tackled in the near future.

Motivation & Objective

• To provide a comprehensive, methodologically focused review of neural network-based constitutive modeling in materials science.
• To classify and compare distinct neural network approaches, including universal approximators, advanced architectures, and physics-integrated models.
• To identify and discuss key challenges in learning constitutive relations, such as extrapolation, interpretability, and error accumulation.
• To highlight the chronological evolution of methods in parallel with advances in machine learning, including CNNs, RNNs, and transformers.
• To advocate for standardized integration of machine learning into commercial simulation software to accelerate interdisciplinary adoption.

Proposed method

• Categorizes neural network approaches into three main classes: universal function approximators, advanced neural network models (e.g., RNNs, CNNs, transformers), and physics-integrated models.
• Reviews training methodologies, including direct and indirect training using experimental data and virtual testing techniques.
• Emphasizes the integration of physical constraints—such as thermodynamic consistency, convexity, and monotonicity—into neural network architectures to improve generalization and reliability.
• Discusses regularization techniques and loss functions that embed physical laws, such as balance laws and entropy inequalities, to guide model learning.
• Analyzes the use of attention mechanisms and transformer architectures for modeling sequential deformation histories with long-range dependencies.
• Proposes that combining data-driven learning with physical priors enhances model robustness, especially in extrapolation and error propagation mitigation.

Figure 1: Number of annual publications (articles, preprints, chapters or proceedings) that contain the combination of the following keywords within the title or the abstract: constitutive and machine learning, constitutive and neural network, material model and neural network, material model and ma

Experimental results

Research questions

• RQ1 How do neural networks serve as universal function approximators in constitutive modeling, and what are their limitations?
• RQ2 What are the advantages and challenges of using advanced neural network architectures (e.g., RNNs, CNNs, transformers) for modeling time- and history-dependent material behavior?
• RQ3 How can physical constraints such as thermodynamic consistency, convexity, and monotonicity be embedded into neural network models to improve generalization and reliability?
• RQ4 What are the key challenges in extrapolation, interpretability, and error accumulation in learned constitutive models, and how can they be addressed?
• RQ5 How can machine learning models be effectively integrated into commercial simulation software to accelerate adoption in engineering applications?

Key findings

• Neural networks can effectively approximate arbitrary continuous constitutive relations, enabling fast surrogate models for complex, nonlinear material behavior.
• Physics-informed neural networks significantly improve model reliability by embedding thermodynamic consistency and physical constraints into the loss function.
• Advanced architectures like transformers show promise in modeling sequential deformation data by capturing long-range dependencies more effectively than RNNs.
• Error accumulation in incremental simulations can be mitigated through regularization and physical constraint enforcement in the training process.
• Interpretability of neural network models improves when physical knowledge is integrated, reducing the 'black-box' nature of deep learning in materials modeling.
• Despite progress, challenges in extrapolation, model evaluation, and computational cost remain critical barriers to widespread adoption in industrial settings.

Figure 2: Methodical spectrum of named constitutive modeling approaches and an outline of the central trade off of these models