[Paper Review] A deep neural network approach on solving the linear transport model under diffusive scaling

Liu Liu, Tieyong Zeng+1 more|arXiv (Cornell University)|Feb 24, 2021

Model Reduction and Neural Networks25 references4 citations

TL;DR

This paper proposes a physics-informed neural network (PINN) framework to solve the linear transport equation under diffusive scaling, a multiscale kinetic equation challenging for traditional numerical methods. By training a deep neural network to satisfy the PDE, initial/boundary conditions, and physical constraints via a composite loss function, the method achieves pointwise convergence to the analytic solution, with theoretical proof that the loss vanishes as the network converges.

ABSTRACT

In this work, we propose a learning method for solving the linear transport equation under the diffusive scaling. Due to the multiscale nature of our model equation, the model is challenging to solve by using conventional methods. We employ the physical informed neural network (PINN) framework, a mesh-free learning method that can numerically solve partial differential equations. Compared to conventional methods (such as finite difference or finite element type), our proposed learning method is able to obtain the solution at any given points in the chosen domain accurately and efficiently, which enables us to better understand the physics underlying the model. In our framework, the solution is approximated by a neural network that satisfies both the governing equation and other constraints. The network is then trained with a combination of different loss terms. Using the approximation theory and energy estimates for kinetic models, we prove theoretically that the total loss vanishes as the neural network converges, upon which the neural network approximated solution converges pointwisely to the analytic solution of the linear transport model. Numerical experiments for two benchmark examples are conducted to verify the effectiveness and accuracy of our proposed method.

Motivation & Objective

• To address the numerical challenges posed by the multiscale nature of the linear transport equation under diffusive scaling.
• To develop a mesh-free, data-driven method that accurately computes solutions at any spatial-temporal-velocity point without discretizing the domain.
• To prove theoretically that the neural network solution converges pointwise to the analytic solution as the loss function vanishes.
• To demonstrate the method's accuracy and robustness on benchmark problems with complex initial and boundary conditions.
• To provide a practical, easy-to-implement alternative to traditional asymptotic-preserving schemes that require deep kinetic theory knowledge.

Proposed method

• The solution is approximated by a deep neural network that embeds the governing PDE, initial condition, and boundary conditions directly into the loss function.
• The loss function combines residuals of the PDE, initial condition, and boundary conditions, with adaptive weights to balance competing loss terms.
• The network is trained via gradient descent to minimize the total loss, leveraging automatic differentiation for efficient backpropagation.
• Theoretical analysis uses approximation theory and energy estimates for kinetic models to prove that vanishing total loss implies pointwise convergence to the true solution.
• A weight balancing algorithm is employed to stabilize training by dynamically adjusting loss term weights based on gradient magnitudes.
• The method is applied to two benchmark problems to validate accuracy and convergence under diffusive scaling.

Experimental results

Research questions

• RQ1 Can a physics-informed neural network effectively solve the linear transport equation under diffusive scaling, a stiff, multiscale kinetic PDE?
• RQ2 Does the PINN framework converge to the true analytic solution as the total loss function approaches zero?
• RQ3 How does the proposed method compare in accuracy and efficiency to conventional finite difference or finite element methods for this problem?
• RQ4 Can the method handle complex initial and boundary conditions without mesh generation or domain truncation?
• RQ5 What is the impact of adaptive loss weighting on training stability and convergence speed?

Key findings

• The total loss function of the PINN framework vanishes as the neural network converges, which is proven using approximation theory and energy estimates for kinetic models.
• The neural network solution converges pointwise to the analytic solution of the linear transport equation when the total loss approaches zero.
• Numerical experiments on two benchmark problems show high accuracy, even under challenging initial and boundary conditions, with the method providing solutions at arbitrary points in the domain.
• The method is mesh-free and avoids the computational burden of traditional methods, especially in high-dimensional velocity spaces or with nonlocal collision operators.
• The adaptive weight balancing algorithm improves training stability and convergence, as demonstrated in the numerical results.
• The approach outperforms conventional methods in handling stiffness and complex geometries without requiring specialized numerical schemes or truncation of the velocity domain.