[Paper Review] HEI: hybrid explicit-implicit learning for multiscale problems

Yalchin Efendiev, Wing Tat Leung+2 more|arXiv (Cornell University)|Sep 5, 2021

Advanced Mathematical Modeling in Engineering26 references4 citations

TL;DR

This paper proposes Hybrid Explicit-Implicit (HEI) learning, a machine learning framework that accelerates multiscale PDE solvers by learning the implicit coarse-scale component ($u_1$) while explicitly computing the fine-scale component ($u_2$). By interpreting the splitting scheme as a sequence-to-sequence translation task, the method uses Transformers to model multiscale dynamics, achieving stable and accurate predictions with relative $L_2$ errors below 0.4% and energy errors below 16% across four numerical examples.

ABSTRACT

Splitting is a method to handle application problems by splitting physics, scales, domain, and so on. Many splitting algorithms have been designed for efficient temporal discretization. In this paper, our goal is to use temporal splitting concepts in designing machine learning algorithms and, at the same time, help splitting algorithms by incorporating data and speeding them up. Since the spitting solution usually has an explicit and implicit part, we will call our method hybrid explicit-implict (HEI) learning. We will consider a recently introduced multiscale splitting algorithms. To approximate the dynamics, only a few degrees of freedom are solved implicitly, while others explicitly. In this paper, we use this splitting concept in machine learning and propose several strategies. First, the implicit part of the solution can be learned as it is more difficult to solve, while the explicit part can be computed. This provides a speed-up and data incorporation for splitting approaches. Secondly, one can design a hybrid neural network architecture because handling explicit parts requires much fewer communications among neurons and can be done efficiently. Thirdly, one can solve the coarse grid component via PDEs or other approximation methods and construct simpler neural networks for the explicit part of the solutions. We discuss these options and implement one of them by interpreting it as a machine translation task. This interpretation successfully enables us using the Transformer since it can perform model reduction for multiple time series and learn the connection. We also find that the splitting scheme is a great platform to predict the coarse solution with insufficient information of the target model: the target problem is partially given and we need to solve it through a known problem. We conduct four numerical examples and the results show that our method is stable and accurate.

Motivation & Objective

• To accelerate contrast-independent splitting schemes for multiscale PDEs by integrating machine learning with physics-based decomposition.
• To address the computational bottleneck of implicit coarse-scale solution ($u_1$) by learning it via data-driven models while computing the explicit component ($u_2$) via PDEs.
• To enable data assimilation for coarse-scale solutions using incomplete or partially observed target models by encoding information from known approximating problems.
• To design hybrid neural architectures that exploit the structural sparsity of explicit components ($u_2$) for efficient learning and reduced communication costs.

Proposed method

• The method decomposes the solution into $u = u_1 + u_2$, where $u_1$ is the coarse-scale component treated implicitly and $u_2$ the fine-scale component treated explicitly, based on multiscale splitting concepts.
• The coarse-scale component $u_1$ is predicted using a Transformer-based model trained on time-series data, treating the splitting scheme as a sequence-to-sequence translation task.
• The explicit component $u_2$ is computed via PDEs or reduced-order models, leveraging its sparse forward map structure for computational efficiency.
• Three strategies are proposed: (1) learning $u_1$ while computing $u_2$, (2) using a hybrid neural network with separate heads for $u_1$ and $u_2$, and (3) learning $u_2$ using data from $u_1$ and reduced models.
• The framework incorporates known physics and data from approximating problems to improve prediction accuracy in coarse solution assimilation tasks.
• The method is validated using four numerical examples, including linear and nonlinear multiscale PDEs, with comparisons to high-fidelity FEM solutions.

Experimental results

Research questions

• RQ1 Can machine learning be effectively integrated into multiscale splitting schemes to accelerate the solution of contrast-independent PDEs?
• RQ2 How can the Transformer architecture be adapted to model the temporal dynamics of coarse-scale components in multiscale problems?
• RQ3 To what extent can learning $u_1$ while explicitly computing $u_2$ reduce computational cost without sacrificing accuracy?
• RQ4 Can the splitting framework serve as a robust platform for coarse solution assimilation with incomplete target model information?
• RQ5 How do hybrid neural architectures that treat $u_1$ implicitly and $u_2$ explicitly compare to standard deep learning models in terms of stability and accuracy?

Key findings

• The HEI learning method achieved a relative $L_2$ error of less than 0.4% for the predicted coarse-scale component $u_{H,1}^n$ in the nonlinear coarse solution assimilation example.
• The energy relative error for the fine-scale solution reconstructed from $u_{H,1}^n + ilde{u}_{H,2}^n$ was reduced to 11.996% on average, compared to 15.498% for $u_{H,1}^n$ alone.
• The relative $L_2$ error for the prediction of $u_{H,1}^n$ remained below 0.01% in the nonlinear example, demonstrating high accuracy in long-term prediction.
• The method successfully reduced the cumulative error in multiscale dynamics by leveraging the Transformer’s ability to encode cross-scale dependencies and model long-range temporal sequences.
• The model achieved stable predictions over 949 time steps in the coarse solution assimilation task, with consistent error profiles and no divergence.
• The use of a separate model for $u_{H,2}^n$ achieved a maximal relative error of less than 0.1% in the nonlinear example, indicating high fidelity for the explicit component.